Re: [PATCH v2] src/port/snprintf.c: Optimize the common base=10 case in fmtint

2021-10-28 Thread Andres Freund
tch (type) setting base, and then separately have branches for the different bases. Greetings, Andres Freund

Re: Minimal logical decoding on standbys

2021-10-28 Thread Andres Freund
al_xlog_page(), contains a bunch of logic > that tries to make sure that we're always reading every record from the > right timeline, but there's nothing similar here. I think that would likely > have to be fixed in order for decoding to work on standbys, but maybe I'm > missing something. I think that part actually works, afaict they both rely on the same XLogReadDetermineTimeline() for that job afaict. What might be missing is logic to update the target timeline. Greetings, Andres Freund

Re: inefficient loop in StandbyReleaseLockList()

2021-10-28 Thread Andres Freund
modifications to the list, or if there's a chance of errors. For the latter there just needs to be a CHECK_FOR_INTERRUPTS() somewhere... Greetings, Andres Freund

Re: inefficient loop in StandbyReleaseLockList()

2021-10-28 Thread Andres Freund
Hi, On 2021-10-28 15:07:48 -0700, Andres Freund wrote: > On 2021-10-28 15:57:51 +0900, Kyotaro Horiguchi wrote: > > I found several other instances of the pattern > > "while(list){list_delete_first(); /*no-break*/}" in > > llvm_release_context, gistProcessEmptyi

Re: inefficient loop in StandbyReleaseLockList()

2021-10-28 Thread Andres Freund
t either - I was more thinking of some of the other cases that deleted the first element, here it's a bit harder to know wether there's a chance of a CFI() or such. Greetings, Andres Freund

parallel vacuum comments

2021-10-30 Thread Andres Freund
much none of this code is heap specific. And vacuumlazy.c is large enough without the parallel code. Greetings, Andres Freund [1] https://postgr.es/m/17245-ddf06aaf85735f36%40postgresql.org

Time to drop plpython2?

2021-10-31 Thread Andres Freund
etely painless, but imo it's not. I was about to list better plpython2/3 support (including the regex replacements for the regress tests) as a TODO for the meson proposal, but it doesn't really seem worth investing in that... Greetings, Andres Freund

Re: emit recovery stats via a new file or a new hook

2021-10-31 Thread Andres Freund
load an extension library first, which users won't have done before hitting the problem. And 3) isn't really possible. I'm not sure that the new log messages aren't sufficient. But if they aren't, it seems better to keep additional data in the stats system, and make them visible via views, rather than adding yet another place to keep stats. Greetings, Andres Freund

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Andres Freund
isn't quite sufficient, because that doesn't tell you whether checkpoints always were triggered by "xlog". > If users need to wait with bated breath for a checkpoint report, we > have something else we need to fix. Well, we probably do. But it's not that easy to fundamentally improve the situation. Greetings, Andres Freund

Re: Time to drop plpython2?

2021-10-31 Thread Andres Freund
Hi, On 2021-10-31 14:49:22 -0400, Tom Lane wrote: > Andres Freund writes: > > To me it seems time to drop plpython2 support. Supporting plpython2 > > until ~7 years after python2 is EOL is already quite long... It'd be one > > thing if it were completely painless, but

Re: should we enable log_checkpoints out of the box?

2021-10-31 Thread Andres Freund
Hi, On 2021-10-31 15:43:57 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2021-10-31 10:59:19 -0400, Tom Lane wrote: > >> No DBA would be likely to consider it as anything but log spam. > > > I don't agree at all. No postgres instance should be run wit

Re: inefficient loop in StandbyReleaseLockList()

2021-10-31 Thread Andres Freund
, I was making that comment in response to the search for other places doing the while(!empty) delete_first() style loops. Some of them are reached via resowners etc, where reaching the same code repeatedly is perhaps more realistic. Greetings, Andres Freund

Use -fvisibility=hidden for shared libraries

2021-10-31 Thread Andres Freund
er & linker to generate a plain function call, rather than having to go through the elf symbol-interposition table. The attached patch is a prototype of this idea. Greetings, Andres Freund [1] https://www.postgresql.org/message-id/20211012083721.hvixq4pnh2pixr3j%40alap3.anarazel.de >From a5

Re: Use -fvisibility=hidden for shared libraries

2021-10-31 Thread Andres Freund
milar to the windows case, which makes it easier to detect build breakages locally... Greetings, Andres Freund

Re: Adding CI to our tree

2021-10-31 Thread Andres Freund
support. I also moved the code main.c code changes to after the CI stuff themselves, as it might be a bit harder to get into a committable shape (at least for me) Greetings, Andres Freund >From 48e921a1838762554780357aacd8a092ed6460f2 Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Mon,

Re: Portability report: ninja

2021-11-01 Thread Andres Freund
xed. I do suspect it's just a python3 issue, as Thomas noted. > Based on these results, I doubt that ninja will give us trouble on any > platform that isn't old enough to get its drivers license. Agreed. There's also alternative compatible ninja implementation in C99 ([1]). But I think it's minimum requirements aren't actually lower than ninja's (it says it requires posix 2008). Greetings, Andres Freund [1] https://github.com/michaelforney/samurai

Re: Eval expression R/O once time (src/backend/executor/execExpr.c)

2021-11-02 Thread Andres Freund
On 2021-11-02 13:43:46 -0400, Tom Lane wrote: > Ranier Vilela writes: > > It seems that 1ec7679f1b67e84be688a311dce234eeaa1d5de8 caused the problem. > > Indeed. Fix pushed. Thanks to both of you!

Re: [PATCH] Native spinlock support on RISC-V

2021-11-02 Thread Andres Freund
Hi, On November 2, 2021 3:55:58 PM PDT, Thomas Munro wrote: >2. When configured with all options on FreeBSD 13, everything looks >good so far except LLVM JIT, which fails with various "Cannot select" >errors. Clang works fine for compiling PostgreSQL itself. Tested >with LLVM 12 (LLVM has sup

Re: [RFC] building postgres with meson -v

2021-11-04 Thread Andres Freund
Hi, On 2021-11-04 19:17:05 +0100, Peter Eisentraut wrote: > On 01.11.21 00:24, Andres Freund wrote: > > - remaining hardcoded configure tests (e.g. ACCEPT_TYPE_ARG*) > > I think we can get rid of that one. Oh, nice! I was somewhat confused by "unsigned int PASCAL"

Re: parallel vacuum comments

2021-11-04 Thread Andres Freund
Hi, On 2021-11-01 10:44:34 +0900, Masahiko Sawada wrote: > On Sun, Oct 31, 2021 at 6:21 AM Andres Freund wrote: > > But even though we have this space optimized bitmap thing, we actually > > need > > more memory allocated for each index, making this whole thing pointles

Re: Time to drop plpython2?

2021-11-04 Thread Andres Freund
t going for plpython would be better long term: Presumably there will be python 4 at some point - but I'd expect that to not be a breaking release, given the disaster that python 3 is. Making a non-versioned name better? Greetings, Andres Freund

Re: should we enable log_checkpoints out of the box?

2021-11-04 Thread Andres Freund
nt violations. Etc. One cannot realistically see LOG or ERROR messages indicating real trouble because we do not provide a realistic way to separate such "normal" log messages from others fairly reliably indicating a problem. > Besides appearing to be unwarranted mockery of what Bharath wrote, Indeed. Greetings, Andres Freund

Re: inefficient loop in StandbyReleaseLockList()

2021-11-04 Thread Andres Freund
t immediately visible from the code, and from the list name one could very well be excused to not be worried about O(N) costs. Greetings, Andres Freund

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-04 Thread Andres Freund
fmgr_info(F_PG_CHECKPOINT, &flinfo); > + > + (void) FunctionCall0Coll(&flinfo, InvalidOid); > + } > break; I don't like this. This turns the checkpoint command which previously didn't rely on the catalog in the happy path etc into something that requires most of the backend to be happily up to work. Greetings, Andres Freund

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-04 Thread Andres Freund
checkpoint predefined role that is only > used for the CHECKPOINT command. What about extending GRANT to allow to grant rights on commands? Yes, it'd be a bit of work to make that work in the catalogs, but it doesn't seem too hard to tackle. Greetings, Andres Freund

Re: WIP: expression evaluation improvements

2021-11-04 Thread Andres Freund
ehow that can be avoided. But maybe CPUs are sufficiently > well-optimized for computing a pointer address as a+b*c that it does not > matter. It should just be a + b, right? Well, for arrays it's more complicated, but it's also more complicated for "normal arrays". > I'm not sure how helpful any of these comments are, but those are my > initial thoughts. It's helpful. The biggest issue I see with getting to the point of actually caching JITed code is the the ->flinfo, ->context thing mentioned above. The best thing I can come up with is moving the allocation of those into the ExprState as well, but my gut says there must be a better approach that I'm not quite seeing. Greetings, Andres Freund

Re: WIP: expression evaluation improvements

2021-11-05 Thread Andres Freund
ers into those arrays of objects. Which then would make the thing non-shareable. Greetings, Andres Freund

Re: WIP: expression evaluation improvements

2021-11-05 Thread Andres Freund
Hi, On 2021-11-05 13:09:10 -0400, Robert Haas wrote: > On Fri, Nov 5, 2021 at 12:48 PM Andres Freund wrote: > > I don't see how that works - the same expression can be evaluated multiple > > times at once, recursively. So you can't have things like > > FunctionC

Re: WIP: expression evaluation improvements

2021-11-05 Thread Andres Freund
Hi, On 2021-11-05 09:48:16 -0700, Andres Freund wrote: > On 2021-11-05 08:34:26 -0400, Robert Haas wrote: > > I'm not sure why that requires all of this relative pointer stuff, > > honestly. Under that problem statement, we don't need everything to be > > one cont

Re: Portability report: ninja

2021-11-05 Thread Andres Freund
On 2021-11-02 09:27:17 +1300, Thomas Munro wrote: > On Tue, Nov 2, 2021 at 8:25 AM Tom Lane wrote: > > 6. While configure.py thinks it knows what to do on AIX, it fails > > on AIX 7.1 and 7.2: > > > > Traceback (most recent call last): > > File "./configure.py", line 544, in > > if platform

Re: WIP: expression evaluation improvements

2021-11-05 Thread Andres Freund
Hi, On 2021-11-05 14:13:38 -0400, Robert Haas wrote: > On Fri, Nov 5, 2021 at 1:20 PM Andres Freund wrote: > > Yes. Optimally we'd do JIT caching across connections as well. One of the > > biggest issues with the costs of JITing is actually parallel query, where > > we&

Re: inefficient loop in StandbyReleaseLockList()

2021-11-06 Thread Andres Freund
* > + * Note that this takes time proportional to the length of the list, > + * since the remaining entries must be moved. > */ > List * > list_delete_first(List *list) Perhaps we could point to list_delete_last()? But it's an improvement without that too. This reminds me that I wanted a list splicing operation for ilist.h... Greetings, Andres Freund

Re: inefficient loop in StandbyReleaseLockList()

2021-11-06 Thread Andres Freund
On 2021-11-06 18:32:54 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2021-11-06 14:06:12 -0400, Tom Lane wrote: > >> + * Note that this takes time proportional to the length of the list, > >> + * since the remaining entries must be moved. > >> */ >

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-07 Thread Andres Freund
On 2021-11-05 08:42:58 -0400, Robert Haas wrote: > On Thu, Nov 4, 2021 at 7:38 PM Jeff Davis wrote: > > It seems like this specific approach has been mostly shot down already. > > But out of curiosity, are you intending to run CHECKPOINT during > > bootstrap or something? > > Imagine a system wi

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-07 Thread Andres Freund
Hi, On 2021-11-05 08:54:37 -0400, Robert Haas wrote: > On Thu, Nov 4, 2021 at 6:46 PM Andres Freund wrote: > > What about extending GRANT to allow to grant rights on commands? Yes, it'd > > be > > a bit of work to make that work in the catalogs, but it doesn't s

Re: Predefined role pg_maintenance for VACUUM, ANALYZE, CHECKPOINT.

2021-11-08 Thread Andres Freund
if one couldn't be found - and doing a full pg_proc read with several subsidiary pg_type reads etc. Greetings, Andres Freund

Re: remove spurious CREATE INDEX CONCURRENTLY wait

2021-11-10 Thread Andres Freund
Hi, On 2020-11-25 17:03:58 -0300, Alvaro Herrera wrote: > On 2020-Nov-23, Andres Freund wrote: > > > On 2020-11-23 12:30:05 -0300, Alvaro Herrera wrote: > > > > In other words, my conclusion is that there definitely is a bug here and > > > I am going to rest

Re: [RFC] building postgres with meson -v

2021-11-10 Thread Andres Freund
Hi, On 2021-11-10 11:07:02 +0100, Peter Eisentraut wrote: > On 01.11.21 00:24, Andres Freund wrote: > > Hi, > > > > Attached is an updated version of the meson patchset. > > Nanoreview: I think the patch Thanks for looking! > Subject: [PATCH v5 11/16] meson: p

Re: Weird failure in explain.out with OpenBSD

2021-11-10 Thread Andres Freund
ng correctly with the necessary tsc offset / speed correction values you'd expect to see something like this. Greetings, Andres Freund

Re: Parallel vacuum workers prevent the oldest xmin from advancing

2021-11-10 Thread Andres Freund
an unnecessary optimization. ProcArrayInstallRestoredXmin() only happens in the context of much more expensive operations. I think it might be worth asserting that the set of flags we're copying is a known subset of the flags that are valid to copy from the source. Greetings, Andres Freund

Re: Interrupts vs signals

2021-11-11 Thread Andres Freund
ices" or whatever the posix language is). Disk IO doesn't count as that. So I don't think it'd be a huge issue. Greetings, Andres Freund

Re: remove spurious CREATE INDEX CONCURRENTLY wait

2021-11-11 Thread Andres Freund
On 2021-11-11 09:38:07 -0300, Alvaro Herrera wrote: > > I just noticed that this commit (dcfff74fb16) didn't revert the change of > > lock > > level in ReplicationSlotRelease(). Was that intentional? > > Hmm, no, that seems to have been a mistake. I'll restore it. Thanks

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
or decided > we're not gonna do it. We need to identify exactly what needs to be > installed before we start pestering the owners. Yea, that's true. Greetings, Andres Freund

Re: JIT doing duplicative optimization?

2021-11-15 Thread Andres Freund
we're doing quadratic effort in the optimization / emission > phase or something like that. Yea. One way to investigate it is to enable jit_dump_bitcode and then use llvm's 'opt' tool to see details about optimization times. opt -time-passes -O3 -o /dev/null < /srv/dev/pgdev-dev/3990041.77.bc |less Greetings, Andres Freund

Re: JIT doing duplicative optimization?

2021-11-15 Thread Andres Freund
; printing them at all? Afaict we don't have detailed docs for EXPLAIN output in general? Greetings, Andres Freund

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
ersion and, iff < 3.0, also output the 'python3' version? Greetings, Andres Freund

Re: [RFC] building postgres with meson

2021-11-15 Thread Andres Freund
Hi, FWIW, I tried building postgres on a few other operating systems using meson, after I got access to the gcc compile farm. Here's the results: - openbsd: Compiled fine. Hit one issue running tests: openbsd has *completely* broken $ORIGIN support. It uses CWD as $ORIGIN rpaths, which obvi

Re: [RFC] building postgres with meson

2021-11-15 Thread Andres Freund
Hi, On 2021-11-15 14:11:25 -0500, Tom Lane wrote: > Andres Freund writes: > > One important thing to call out: Meson has support for the AIX linker, but > > *not* the xlc compiler. I.e. one has to use gcc (or clang, but I didn't > > try). I don't know if we&#

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
ions the AIX toolbox has 3.7.11. I don't know enough about AIX to know whether there's other sources of python3, bison, etc that are common. On the AIX system I have access to they all seem to be symlinked to /opt/freeware, which I understand is that toolbox stuff? Greetings, Andres Freund

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
rt before PG15 ships, so maybe we don't need to > worry about whether we can build with their default toolsets. > Still, it's a tradeoff I'd rather not make. There's two other paths worth mentioning: - muon is a WIP alternative implementation of meson in plain C99. Doesn't yet have enough feature coverage. - pyinstaller generates an executable from the meson python code, and supports solaris and aix Greetings, Andres Freund

Re: Time to drop plpython2?

2021-11-15 Thread Andres Freund
Hi, On November 15, 2021 12:36:11 PM PST, Robert Haas wrote: >It'd only be an issue if they want to compile from source, right? >We're not speaking of changing the runtime prerequisites, IIUC. Correct. -- Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: [RFC] building postgres with meson

2021-11-15 Thread Andres Freund
ietary ones, to the > extent that the latter will be dead soon anyway? I think that's a pretty clear trend. The ones that aren't dying seem to be incrementally onto more and more rebasing onto llvm tooling. It doesn't help that most of those compilers are primarily for OSs that, uh, aren't exactly growing. Which limits their potential usability significantly. Greetings, Andres Freund

Re: RecoveryInProgress() has critical side effects

2021-11-16 Thread Andres Freund
elevant platforms (at least once the aarch64 thing is fixed, there's a patch). XLogCtlInsert already takes care to ensure that RedoRecPtr/fullPageWrites are on a cacheline that's not constantly modified. If we really wanted to optimize the no-8-byte-single-copy-atomicity path, we could introduce a counter counting how many times RedoRecPtr/fullPageWrites have changed. But it seems unlikely to be worth it. Greetings, Andres Freund

Re: RecoveryInProgress() has critical side effects

2021-11-16 Thread Andres Freund
ment latencies in milliseconds: 0.125 SELECT 1; 0.024 SELECT 1; 0.095 SELECT 'txid_current()'::regprocedure; 0.025 SELECT 'txid_current()'::regprocedure; 0.033 SELECT txid_current(); 0.024 SELECT txid_current(); which nicely shows how much of the overhead is not related to the actual txid_current() invocation. Greetings, Andres Freund

Re: RecoveryInProgress() has critical side effects

2021-11-16 Thread Andres Freund
d I didn't see any meaningful and repeatable performance difference 354a1f8d220, ad26ee28250 and 0002 applied ontop of ad26ee28250. The run-to-run variance is way higher than the difference between the changes. Greetings, Andres Freund

Re: RecoveryInProgress() has critical side effects

2021-11-16 Thread Andres Freund
On 2021-11-16 16:30:27 -0500, Robert Haas wrote: > I'm still not entirely clear on whether you prefer v1-0002, v2-0002, > or something else. I think it basically doesn't matter much. It's such a small piece of the cost compared to either the cost of a single insert or the ratio between RedoRecPtr/

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Andres Freund
uplicate and how many places to change to choose the right flag variable. It's taken a while for this to become a real issue, so it perhaps was the right choice at the time. Greetings, Andres Freund

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Andres Freund
as you are using an accache file it's not really going > to cost that much. (BTW, does meson have any comparable optimization? > If it doesn't, I bet that is going to be a problem.) > > regards, tom lane Greetings, Andres Freund

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Andres Freund
cluding the commandline (i.e. compiler flags) as the cache key. So no more errors about compile flags changing... Greetings, Andres Freund

Re: Mixing CC and a different CLANG seems like a bad idea

2021-11-18 Thread Andres Freund
Hi, On 2021-11-18 16:13:50 -0500, Robert Haas wrote: > On Thu, Nov 18, 2021 at 3:43 PM Andres Freund wrote: > > Hi, > > > > Greetings, > > > > Andres Freund > > Greetings to you too, Andres. :-) Oops I sent the email that I copied text from, rather than

Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)

2021-11-19 Thread Andres Freund
| 4 +- > src/include/utils/backend_status.h | 80 ++ > src/test/regress/expected/rules.out | 8 + > 16 files changed, 701 insertions(+), 31 deletions(-) This is a pretty large change, I wonder if there's a way to make it a bit more granular. Greetings, Andres Freund

Re: update with no changes

2021-11-19 Thread Andres Freund
Hi, On November 19, 2021 8:38:25 AM PST, Marcos Pegoraro wrote: >Why this happens ? > >create table t(i int); >CREATE TABLE >insert into t values(1); >INSERT 0 1 >select (ctid::text::point)[1]::int, * from t; > ctid | i >--+--- >1 | 1 >(1 row) >update t set i = i; >UPDATE 1 >select (ctid

Re: Adding CI to our tree

2021-11-19 Thread Andres Freund
et clean > + apt-get clean && \ > + rm -f /var/lib/apt/lists/* Might not matter too much compared to the size of the whole thing, but it definitely won't hurt... Greetings, Andres Freund

Re: TOAST - why separate visibility map

2021-11-19 Thread Andres Freund
Hi, On November 19, 2021 12:31:00 PM PST, Tom Lane wrote: >Virender Singla writes: >> Why do Toast tables have it's own visibility map and xmin, xmax columns etc? >> Isn't it increasing row size in a toast table and adding more complexity? Given the size of toasted data, the overhead is unlike

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-11-22 Thread Andres Freund
oes it make sense to track DEAD tuples this way? Isn't that going to lead to counting them over-and-over again? I think it's quite misleading to include them in "dead bot not yet removable". > + /* > + * Now save details of the LP_DEAD items from the page in the > dead_tuples > + * array iff VACUUM uses two-pass strategy case > + */ Do we really need to have separate code for this in lazy_scan_prune() and lazy_scan_noprune()? > + } > + else > + { > + /* > + * We opt to skip FSM processing for the page on the grounds > that it > + * is probably being modified by concurrent DML operations. > Seems > + * best to assume that the space is best left behind for future > + * updates of existing tuples. This matches what opportunistic > + * pruning does. Why can we assume that there concurrent DML rather than concurrent read-only operations? IME it's much more common for read-only operations to block cleanup locks than read-write ones (partially because the frequency makes it easier, partially because cursors allow long-held pins, partially because the EXCLUSIVE lock of a r/w operation wouldn't let us get here) I think this is a change mostly in the right direction. But as formulated this commit does *WAY* too much at once. Greetings, Andres Freund

Reduce function call costs on ELF platforms

2021-11-22 Thread Andres Freund
48 89 4d c0 mov%rcx,-0x40(%rbp) 2838f1: 49 8d 3c 17 lea(%r15,%rdx,1),%rdi 2838f5: 48 89 55 c8 mov%rdx,-0x38(%rbp) 2838f9: ff 15 09 45 66 00 call *0x664509(%rip)# 8e7e08 I haven't run detailed benchm

Re: Reduce function call costs on ELF platforms

2021-11-22 Thread Andres Freund
FO_V1). For extensions that are referenced that way that actually seems like a good thing, because it makes it clearer what could be referenced that way. Greetings, Andres Freund

Re: logical decoding and replication of sequences

2021-11-22 Thread Andres Freund
27;s simply no evidence of sequence use in a transaction if that transaction uses a previously logged sequence value. Greetings, Andres Freund

Re: Reduce function call costs on ELF platforms

2021-11-22 Thread Andres Freund
Hi, On 2021-11-22 20:13:28 -0500, Bruce Momjian wrote: > On Mon, Nov 22, 2021 at 03:57:45PM -0800, Andres Freund wrote: > > It does however change symbol binding, basically making all symbols bound > > eagerly. Which I guess theoretically could be considered an ABI change, > &g

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-11-22 Thread Andres Freund
s often legit to read a heap page without a buffer lock > (only a pin), I can't see why BufferGetPage() without a buffer lock > shouldn't also be okay -- if anything it seems safer. I think that I > would agree with you if it wasn't for that inconsistency (which is > rather a big "if", to be sure -- even for me). At least for heap it's rarely legit to read buffer contents via BufferGetPage() without a lock. It's legit to read data at already-determined offsets, but you can't look at much other than the tuple contents. > > Why does it make sense to track DEAD tuples this way? Isn't that going to > > lead > > to counting them over-and-over again? I think it's quite misleading to > > include > > them in "dead bot not yet removable". > > Compared to what? Do we really want to invent a new kind of DEAD tuple > (e.g., to report on), just to handle this rare case? When looking at logs I use the "tuples: %lld removed, %lld remain, %lld are dead but not yet removable, oldest xmin: %u\n" line to see whether the user is likely to have issues around an old transaction / slot / prepared xact preventing cleanup. If new_dead_tuples doesn't identify those cases anymore that's not reliable anymore. > I accept that this code is lying about the tuples being RECENTLY_DEAD, > kind of. But isn't it still strictly closer to the truth, compared to > HEAD? Counting it as RECENTLY_DEAD is far closer to the truth than not > counting it at all. I don't see how it's closer at all. There's imo a significant difference between not being able to remove tuples because of the xmin horizon, and not being able to remove it because we couldn't get a cleanup lock. Greetings, Andres Freund

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

2021-11-23 Thread Andres Freund
all-frozen pages is small? We could perhaps do better with if we had information about the system-wide rate of xid throughput and how often / how long past vacuums of a table took. Greetings, Andres Freund

Re: Reduce function call costs on ELF platforms

2021-11-24 Thread Andres Freund
uses of LD_PRELOAD. In particular, it doesn't break things like replacing the malloc implementation. When do you have a symbol that you want to override *inside* your library (executables already bind to their own symbols at compile time)? I've seen that for replacing buggy functions in closed source things, but that's about it? Greetings, Andres Freund

Re: Reduce function call costs on ELF platforms

2021-11-24 Thread Andres Freund
Hi, On 2021-11-24 17:55:03 -0500, Andrew Dunstan wrote: > On 11/24/21 13:55, Andres Freund wrote: > > On 2021-11-23 17:28:08 +0100, Peter Eisentraut wrote: > >> On 22.11.21 23:32, Tom Lane wrote: > >>>> The easier approach for this class of issues is to use th

Re: Separate out FileSet from SharedFileSet (was Re: pgsql: pgstat: Bring up pgstat in BaseInit() to fix uninitialized use o)

2021-11-29 Thread Andres Freund
Hi, Thank for all the work on this topic - I'd somehow accidentally marked this thread as read when coming back from vacation... Greetings, Andres Freund

Re: Deprecating the term "super-exclusive lock"

2021-12-01 Thread Andres Freund
> "super-exclusive lock" from the tree completely. The actual relevant C > symbols only use the term cleanup. +1 Greetings, Andres Freund

Re: pg_replslotdata - a tool for displaying replication slot information

2021-12-01 Thread Andres Freund
#x27;s sufficient need for this. > I don't have any other compelling use- cases at the moment, but I will say > that it is typically nice from an administrative standpoint to be able to > inspect things like this without logging into a running server. Weighed against the cost of maintaining (including documentation) a separate tool this doesn't seem sufficient reason. Greetings, Andres Freund

Re: O(n) tasks cause lengthy startups and checkpoints

2021-12-01 Thread Andres Freund
unnecessary snapshot files, rather than making the cleanup more asynchronous. Greetings, Andres Freund

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

2018-03-23 Thread Andres Freund
On 2018-03-23 11:06:48 +, Simon Riggs wrote: > Your assumption that I would commit a new patch that was 29 mins old > is frankly pretty ridiculous, so yes, lets keep calm. Uh.

Re: max_memory_per_backend GUC to limit backend's memory usage

2018-03-23 Thread Andres Freund
an be more robust. Configuring overcommit_memory to not overcommit should do the trick. Greetings, Andres Freund

Re: Backend memory dump analysis

2018-03-23 Thread Andres Freund
luding in-core? The overhead required for it (in cycles, in higher memory usage due to additional bookeeping, in maintenance) makes me highly doubtful it's worth going there. While I definitely can see the upside, it doesn't seem to justify the cost. Greetings, Andres Freund

Re: Backend memory dump analysis

2018-03-23 Thread Andres Freund
On 2018-03-23 18:05:38 +, Vladimir Sitnikov wrote: > Andres>The overhead required for it (in cycles, in higher memory usage due > to > additional bookeeping > > Does that mean the memory contexts are unparseable? (there's not enough > information to enumerate contents) You can enumerate them

Re: ppc64le support in 9.3 branch?

2018-03-23 Thread Andres Freund
o it. I'm inclined to just ask them to stop running the animals on that branch. There are no pre-existing users on 9.3 ppc64le, and new customers hopefully won't move to 9.3. ISTM backpatching is riskier than just changing a bunch of buildfarm configurations. Greetings, Andres Freund

Re: Backend memory dump analysis

2018-03-23 Thread Andres Freund
ople have to adapt their code because of 9fa6f00b1 breaking > usages with nonconstant context names, they have a solution to turn to > immediately rather than having to change things again in v12. Yea, that'd make sense. Greetings, Andres Freund

Re: Backend memory dump analysis

2018-03-23 Thread Andres Freund
On 2018-03-23 15:12:43 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-03-23 14:33:25 -0400, Tom Lane wrote: > >> + MemoryContextSetIdentifier(func_cxt, function->fn_signature); > >> > >> This would cost an extra char * field in struct Me

Re: Undesirable entries in typedefs list

2018-03-24 Thread Andres Freund
quot;bool", we could probably deal with that most reliably by > having pgindent add it as a special case. Maybe we could get it > back in there by having some trailing-edge buildfarm member > contribute typedefs, but that seems like a solution with a rather > limited half-life. Could we combine the list of typedefs with one manually maintained in-tree? Greetings, Andres Freund

Re: Undesirable entries in typedefs list

2018-03-24 Thread Andres Freund
in that area won't be known. The number of typedefs we actually use is fairly small (~5-7), so we could realistically maintain the them manually. I'm about to head out, but afterwards I'm going to check what the typedefs collected actually are when LLVM is enabled. Greetings, Andres Freund

Re: Why does load_external_function() return PGFunction?

2018-03-24 Thread Andres Freund
On 2018-02-06 15:43:29 -0500, Tom Lane wrote: > Andres Freund writes: > > We've several callers to load_external_function() that do not use the > > returned value as a PGFunction. I'd vote for changing the return type to > > void * and have fmgr.c cast it to

Re: Undesirable entries in typedefs list

2018-03-24 Thread Andres Freund
On 2018-03-24 10:41:40 -0700, Andres Freund wrote: > I'm about to head out, but afterwards I'm going to check what the > typedefs collected actually are when LLVM is enabled. It's indeed from llvm: <3><5201>: Abbrev Number: 4 (DW_TAG_typedef) <5202>

Re: pgsql: Avoid premature free of pass-by-reference CALL arguments.

2018-03-24 Thread Andres Freund
notation like 8.26204195618629e-05. So, perhaps we should choose a different volatile function here? Greetings, Andres Freund

Re: Undesirable entries in typedefs list

2018-03-24 Thread Andres Freund
On 2018-03-24 14:51:32 -0700, Peter Geoghegan wrote: > On Sat, Mar 24, 2018 at 1:36 PM, Andres Freund wrote: > > I've attached the difference between a objdump typedefs list roughly > > equivalent to what the buildfarm uses. There's no difference when not > >

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Andres Freund
On 2018-03-26 11:14:03 -0400, Robert Haas wrote: > On Sat, Mar 24, 2018 at 4:33 PM, Andres Freund wrote: > > I don't think it really changes the need, but it's worthwhile to note > > that gcc-8 warns about this now: > > /home/andres/src/postgresql/src/backend/pos

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Andres Freund
ore function pointers in datums, that ship has sailed a long while ago, no? Greetings, Andres Freund

Re: Proposal: http2 wire format

2018-03-26 Thread Andres Freund
anges to v3, or start a multiyear discussion on the design of > the next protocol. And you’ll still converge to something like HTTP2 > or QUIC. *NONE* of the interesting problems are solved by HTTP2. You *still* need a full blown protocol ontop of it. So no, this doesn't change that. Greetings, Andres Freund

Re: JIT compiling with LLVM v12.2

2018-03-26 Thread Andres Freund
ferent heap allocators (the same reason we provide > PQfreemem in libpq). I just kept it similar to nearby functions in the LLVM code. > Fix brace style. I tried to keep this as it's submitted to LLVM, I hope we can get rid of them for newer version soon... I think I'll update them to be exactly the same as soon as the upstream patch is applied. Greetings, Andres Freund

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Andres Freund
On 2018-03-26 15:26:00 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-02-06 15:43:29 -0500, Tom Lane wrote: > >> void* isn't necessarily compatible with function pointers --- there are > >> platforms where they're physically different widths, thoug

Re: Why does load_external_function() return PGFunction?

2018-03-26 Thread Andres Freund
On 2018-03-26 11:31:36 -0700, Andres Freund wrote: > On 2018-03-26 11:14:03 -0400, Robert Haas wrote: > > On Sat, Mar 24, 2018 at 4:33 PM, Andres Freund wrote: > > > I don't think it really changes the need, but it's worthwhile to note > > > that gcc-8 warns a

Re: [HACKERS] Partition-wise aggregation/grouping

2018-03-26 Thread Andres Freund
Hi, On 2018-03-23 17:01:54 +0530, Jeevan Chalke wrote: > Attached patch which fixes that. Thanks, will push. For the future, I'd be more likely to notice if you CC me ;) > However, I am not sure whether it is expected to have stable regression run > with installcheck having local settings. > Fo

Re: Parallel Aggregates for string_agg and array_agg

2018-03-26 Thread Andres Freund
parallelism, the other doesn't, without duplicating the backing code... Not pretty tho. Greetings, Andres Freund

Re: JIT compiling with LLVM v12

2018-03-27 Thread Andres Freund
On 2018-03-27 10:05:47 -0400, Peter Eisentraut wrote: > On 3/13/18 19:40, Andres Freund wrote: > > I've pushed a revised and rebased version of my JIT patchset. > > What is the status of this item as far as the commitfest is concerned? 7/10 committed. Inlining, Explain, Doc

Re: [HACKERS] logical decoding of two-phase transactions

2018-03-27 Thread Andres Freund
atch 4 so that this is a whole, > > committable patch > > * added comments to make abort and commit structs look same > > > > Attached patch is proposed for a separate, early commit as part of this > > Looking to commit "logging GID" patch today, if no further objections. None here. Greetings, Andres Freund

<    5   6   7   8   9   10   11   12   13   14   >