Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Thomas Munro
On Thu, May 3, 2018 at 5:18 PM, Thomas Munro wrote: > 2018-05-03 05:07:25.904 UTC [19677] DEBUG: using armv8 crc2 hardware = 1 Let me try that again with that stupid typo (crc2) fixed... -- Thomas Munro http://www.enterprisedb.com 0001-Fix-endianness-bug-in-ARMv8-CRC32-detection-v2.patch Des

Re: Issues while building PG in MS Windows, using MSYS2 and MinGW-w64

2018-05-02 Thread Alexander Lakhin
"insaf.k" wrote: I am trying to build PG from source, in MS Windows using MSYS2 and MinGW-w64. I've tried to build PG 10.0 as wells as 10.3. I've done configuring like this ./configure --prefix="/d/pg10/" And when I do "make" or "make world", I'm getting compilation error. I've attached comple

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Thomas Munro
On Thu, May 3, 2018 at 4:48 PM, Tom Lane wrote: > Thomas Munro writes: >> On Thu, May 3, 2018 at 4:04 PM, Tom Lane wrote: >>> It strikes me also that, at least for debugging purposes, it's seriously >>> awful that you can't tell from outside what result this function got. > >> I don't think *bro

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Tom Lane
Thomas Munro writes: > On Thu, May 3, 2018 at 4:04 PM, Tom Lane wrote: >> It strikes me also that, at least for debugging purposes, it's seriously >> awful that you can't tell from outside what result this function got. > I don't think *broken* CPUs are something we need to handle, are they? I'

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Thomas Munro
On Thu, May 3, 2018 at 4:04 PM, Tom Lane wrote: > I wrote: >> Andrew Gierth writes: >>> Isn't there a hidden assumption about endianness there? Right, thanks. >> Yeah, I was wondering about that too, but does anyone actually run >> ARMs big-endian? I think all the distros I follow dropped big

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Tom Lane
I wrote: > Andrew Gierth writes: >> Isn't there a hidden assumption about endianness there? > Yeah, I was wondering about that too, but does anyone actually run > ARMs big-endian? After a bit more thought ... we could remove the magic constant, along with any assumptions about endianness, by doi

Re: GSoC 2018: thrift encoding format

2018-05-02 Thread Charles Cui
Thanks for your confirm Aleksander! Also I am thinking of how to deal with complex data structure like map, list, or set. I guess one possible solution is to get raw data bytes for these data structure? Otherwise it could be hard to wrap into a Datum. 2018-05-02 8:38 GMT-07:00 Aleksander Alekseev

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Tom Lane
Andrew Gierth writes: > "Thomas" == Thomas Munro writes: > + uint64 data = 42; > Isn't there a hidden assumption about endianness there? Yeah, I was wondering about that too, but does anyone actually run ARMs big-endian? regards, tom lane

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Andrew Gierth
> "Thomas" == Thomas Munro writes: + uint64 data = 42; Isn't there a hidden assumption about endianness there? -- Andrew (irc:RhodiumToad)

Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread David G. Johnston
On Wednesday, May 2, 2018, Alvaro Herrera wrote: > Robert Haas wrote: > > On Wed, May 2, 2018 at 9:28 AM, Alvaro Herrera > wrote: > > > I admit I am more concerned about the possibility of bugs than I am > > > about providing a performance-related tool. > > > > I agree that if partition pruning

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, May 2, 2018 at 9:28 AM, Alvaro Herrera > wrote: > > I admit I am more concerned about the possibility of bugs than I am > > about providing a performance-related tool. > > I agree that if partition pruning has bugs, somebody might want to > turn it off. On the other

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Yuriy Zhuravlev
> > Sorry, but comparing lines at that state is just bullshit. I totally disagree, proportions will be same in any case. Most of the comments of converted tests are missing. > Add 100-500 lines? ok. You detect like a third of the things that the old configure > detected. > I tried to use CMak

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Andres Freund
On 2018-05-03 09:42:49 +0900, Yuriy Zhuravlev wrote: > 2018-05-03 9:32 GMT+09:00 Andres Freund : > > Given that you don't have feature parity this just seems like trolling. > > > > I have. I have some lacks with .po generation and documentation but all! > other features same, I even can run tap te

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread Robert Haas
On Wed, May 2, 2018 at 9:28 AM, Alvaro Herrera wrote: > I admit I am more concerned about the possibility of bugs than I am > about providing a performance-related tool. I agree that if partition pruning has bugs, somebody might want to turn it off. On the other hand, when they do, there's a goo

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Yuriy Zhuravlev
2018-05-03 9:32 GMT+09:00 Andres Freund : > On 2018-05-03 09:29:32 +0900, Yuriy Zhuravlev wrote: > > > > > > I don't think that unsubstantiated hyperbole is the right way to > > > approach the task of convincing the community to adopt the approach > > > you prefer. > > > > > > It's not a hyperbole

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Andres Freund
On 2018-05-03 09:29:32 +0900, Yuriy Zhuravlev wrote: > > > > I don't think that unsubstantiated hyperbole is the right way to > > approach the task of convincing the community to adopt the approach > > you prefer. > > > It's not a hyperbole it's fact and I even talked about it on conference. > Yo

Re: Oddity in tuple routing for foreign partitions

2018-05-02 Thread Robert Haas
On Wed, May 2, 2018 at 7:06 AM, Etsuro Fujita wrote: > Here is a small patch to remove a no-longer-needed cast in > postgresBeginForeignInsert(). Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Yuriy Zhuravlev
> > I don't think that unsubstantiated hyperbole is the right way to > approach the task of convincing the community to adopt the approach > you prefer. It's not a hyperbole it's fact and I even talked about it on conference. You should just compare all my cmake files with Makefile+.in+.m4 (and m

Re: Unportable code in autoprewarm.c

2018-05-02 Thread Robert Haas
On Wed, May 2, 2018 at 3:21 PM, Tom Lane wrote: > Is there a reason why this record count needs to be int64 rather than > plain int, and if so what? This code is not exactly well documented, > but it looks to me like the number of records should be bounded by > NBuffers, which is an int and is un

Re: Is there a memory leak in commit 8561e48?

2018-05-02 Thread Michael Paquier
On Wed, May 02, 2018 at 07:03:21PM -0400, Tom Lane wrote: > It's only ~100 bytes per stack level. I think under normal loads > nobody would notice. If you're worried about cross-transaction > memory consumption, our various caches tend to be a lot worse. Perhaps, that's one reason why people dro

Re: A few warnings on Windows

2018-05-02 Thread Michael Paquier
On Thu, May 03, 2018 at 09:03:15AM +1200, Thomas Munro wrote: > On Thu, May 3, 2018 at 8:01 AM, Tom Lane wrote: > Build succeeded. > 0 Warning(s) > 0 Error(s) > > Huzzah! Great work. This has been some time... -- Michael signature.asc Description: PGP signature

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread David Rowley
On 3 May 2018 at 11:38, David G. Johnston wrote: > Maybe "Partition Filtering" (I'm disliking selection, I'm thinking we must > always select partitions) I don't see why "Filtering" is any different from pruning, they both imply removing something that was once there. What I'm saying is, that it'

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread David G. Johnston
On Wed, May 2, 2018 at 4:06 PM, David Rowley wrote: > On 1 May 2018 at 21:44, Amit Langote > wrote: > > About the patch in general, it seems like the newly added documentation > > talks about "Partition Pruning" as something that *replaces* constraint > > exclusion. But, I think "Partition Prun

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Thomas Munro
On Thu, May 3, 2018 at 10:08 AM, Tom Lane wrote: > I don't have any way to test this, but it looks plausible, so pushed. > > (I note you forgot to run autoheader, btw.) Thanks! -- Thomas Munro http://www.enterprisedb.com

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread David Rowley
On 1 May 2018 at 21:44, Amit Langote wrote: > About the patch in general, it seems like the newly added documentation > talks about "Partition Pruning" as something that *replaces* constraint > exclusion. But, I think "Partition Pruning" is not the thing that > replaces constraint exclusion. Jus

Re: Is there a memory leak in commit 8561e48?

2018-05-02 Thread Tom Lane
Michael Paquier writes: > With connection poolers letting the connections to the server be around > for a long time, wouldn't it be an issue to let this much memory live > longer than the transaction context? The deeper the stack, the more > memory consumed, hence the more OS cache that PostgreSQ

Re: Is there a memory leak in commit 8561e48?

2018-05-02 Thread Michael Paquier
On Wed, May 02, 2018 at 05:20:37PM -0400, Tom Lane wrote: > Peter Eisentraut writes: > > Yes, that was the idea. Here is an adjusted patch. > > Looks OK to me as far as the leak issue goes. I have no opinion on > whether this is adequate in respect to cleanup-after-error problems. With connect

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Catalin Iacob
On Wed, May 2, 2018 at 5:44 PM, Robert Haas wrote: > I don't think that unsubstantiated hyperbole is the right way to > approach the task of convincing the community to adopt the approach > you prefer. I don't see that any compelling evidence has been > presented that a cmake-based solution would

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Tom Lane
Andres Freund writes: > Now you could argue that we could just rewrite to non-recursive > make. But that'd be nearly as much work as migrating to another > buildsystem. I'm sure it'd be a significant amount of work ... but it wouldn't require redesigning any configuration or portability hacks, no

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Tom Lane
Thomas Munro writes: > On Thu, May 3, 2018 at 2:30 AM, Tom Lane wrote: >> Do you really need the pg_crc32c_armv8_choose_dummy global variable? > True. Of course I needed an interesting length too... I don't have any way to test this, but it looks plausible, so pushed. (I note you forgot to ru

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Andres Freund
On 2018-05-02 23:43:50 +0200, Hartmut Holzgraefe wrote: > On 02.05.2018 17:44, Robert Haas wrote: > > But having parallel make work better and more efficiently > > and with fewer hard-to-diagnose failure modes would definitely be > > nice. > > that's especially a thing I haven't seen in "our" envi

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Thomas Munro
On Thu, May 3, 2018 at 2:30 AM, Tom Lane wrote: > Do you really need the pg_crc32c_armv8_choose_dummy global variable? > That seems pretty ugly. If you're concerned about the compiler > optimizing away the call to the crc function, you could write it like > > result = (pg_comp_crc32c_armv8(0,

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Hartmut Holzgraefe
On 02.05.2018 17:44, Robert Haas wrote: But having parallel make work better and more efficiently and with fewer hard-to-diagnose failure modes would definitely be nice. that's especially a thing I haven't seen in "our" environment, this was an area where autotools and cmake didn't really diffe

Re: Is there a memory leak in commit 8561e48?

2018-05-02 Thread Tom Lane
Peter Eisentraut writes: > Yes, that was the idea. Here is an adjusted patch. Looks OK to me as far as the leak issue goes. I have no opinion on whether this is adequate in respect to cleanup-after-error problems. regards, tom lane

Re: A few warnings on Windows

2018-05-02 Thread Tom Lane
Thomas Munro writes: > One more problem. whelk builds against Python 3.6 and says: > c:\users\pgbf\appdata\local\programs\python\python36-32\include\pyconfig.h(174): > warning C4142: benign redefinition of type > (src/pl/plpython/plpy_elog.c) > [C:\buildfarm\buildenv\HEAD\pgsql.build\plpython3.v

Re: A few warnings on Windows

2018-05-02 Thread Thomas Munro
On Thu, May 3, 2018 at 8:01 AM, Tom Lane wrote: > Seems reasonable to me. Pushed, we'll see what the buildfarm thinks. Build succeeded. 0 Warning(s) 0 Error(s) Huzzah! One more problem. whelk builds against Python 3.6 and says: c:\users\pgbf\appdata\local\programs\python\python36-32\

Re: Is there a memory leak in commit 8561e48?

2018-05-02 Thread Peter Eisentraut
On 5/2/18 12:30, Tom Lane wrote: > I'm not particularly. It seems impossible that _SPI_stack could grow > faster than the machine's actual stack, which would mean (on typical > setups) that you couldn't get more than perhaps 10MB of _SPI_stack > before hitting a stack-overflow error. That's a lot

Re: A few warnings on Windows

2018-05-02 Thread Tom Lane
Thomas Munro writes: > The only remaining warnings on those machines now come from the fact > that our port_win32.h and Perl's XSUB.h both want to define macros to > define macros for libc functions like mkdir etc. plperl.h already > seems to deal with other similar kinds of ugliness. Perhaps if

Unportable code in autoprewarm.c

2018-05-02 Thread Tom Lane
I've been going through compiler warnings from the buildfarm, and I notice jacana is unhappy about this: /* First line of the file is a record count. */ if (fscanf(file, "<<" INT64_FORMAT ">>\n", &num_elements) != 1) It's entirely correct to complain, because we only guarantee tha

Re: stats_ext test fails with -DCATCACHE_FORCE_RELEASE

2018-05-02 Thread Andrew Dunstan
On Wed, May 2, 2018 at 11:48 AM, Tom Lane wrote: > tl;dr: we lack buildfarm coverage of CATCACHE_FORCE_RELEASE. > This is probably bad. It might be okay to just add that to > prion's configuration; Will do that pronto. cheers andre -- Andrew Dunstanhttps://www.2ndQuadrant.co

Re: [HACKERS] Clock with Adaptive Replacement

2018-05-02 Thread Vladimir Sitnikov
>If somebody produced a trace showing the buffer lookups in order To get things moving, I've created a DTrace script that captures buffer reads: https://github.com/vlsi/pgsqlstat/blob/pgsqlio/pgsqlio Is it something that can be used to capture live traces? Sample output can be seen here: https:/

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
On Wed, May 2, 2018 at 11:06 AM, Tom Lane wrote: >> -1 from me. What about the case where only some tuples are massive? > > Well, what about it? If there are just a few wide tuples, then the peak > memory consumption will depend on how many of those happen to be in memory > at the same time ... b

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Tom Lane
Peter Geoghegan writes: > On Wed, May 2, 2018 at 10:43 AM, Heikki Linnakangas wrote: >> Independently of this, perhaps we should put in special case in >> dumptuples(), so that it would never create a run with fewer than maxTapes >> tuples. The rationale is that you'll need to hold that many tupl

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
On Wed, May 2, 2018 at 10:43 AM, Heikki Linnakangas wrote: > I'm not sure what you could derive that from, to make it less arbitrary. At > the moment, I'm thinking of just doing something like this: > > /* > * Minimum amount of memory reserved to hold the sorted tuples in > * TSS_BUILDRUNS phase

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
On Wed, May 2, 2018 at 10:43 AM, Heikki Linnakangas wrote: > Independently of this, perhaps we should put in special case in > dumptuples(), so that it would never create a run with fewer than maxTapes > tuples. The rationale is that you'll need to hold that many tuples in memory > during the merg

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Heikki Linnakangas
On 02/05/18 19:41, Tom Lane wrote: Robert Haas writes: On Wed, May 2, 2018 at 11:38 AM, Heikki Linnakangas wrote: To fix, I propose that we change the above so that we always subtract tapeSpace, but if there is less than e.g. 32 kB of memory left after that (including, if it went below 0), th

GSOC 2018

2018-05-02 Thread Joshua D. Drake
-hackers, Who is coordinating GSOC this year? Thanks, JD -- Command Prompt, Inc. || http://the.postgres.company/ || @cmdpromptinc PostgreSQL centered full stack support, consulting and development. Advocate: @amplifypostgres || Learn: https://postgresconf.org * Unless otherwise stat

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
On Wed, May 2, 2018 at 8:38 AM, Heikki Linnakangas wrote: > With a small work_mem values, maxTapes is always 6, so tapeSpace is 48 kB. > With a small enough work_mem, 84 kB or below in this test case, there is not > enough memory left at this point, so we don't subtract tapeSpace. However, > with

Re: Transform for pl/perl

2018-05-02 Thread Dagfinn Ilmari Mannsåker
Peter Eisentraut writes: > These two items are now outstanding: > > On 4/10/18 07:33, Dagfinn Ilmari Mannsåker wrote: >> 2) jsonb scalar values are passed to the plperl function wrapped in not >>one, but _two_ layers of references > > I don't understand this one, or why it's a problem, or wha

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Tom Lane
Robert Haas writes: > On Wed, May 2, 2018 at 11:38 AM, Heikki Linnakangas wrote: >> To fix, I propose that we change the above so that we always subtract >> tapeSpace, but if there is less than e.g. 32 kB of memory left after that >> (including, if it went below 0), then we bump availMem back up

Re: Is there a memory leak in commit 8561e48?

2018-05-02 Thread Tom Lane
Peter Eisentraut writes: > On 5/1/18 11:42, Tom Lane wrote: >> Peter Eisentraut writes: >> That seems like an odd way to approach this. Why not just remove the >> reset of _SPI_stack and _SPI_stack_depth, so as to subtract code rather >> than adding it --- that is, make it actually work like you

Re: [HACKERS] Clock with Adaptive Replacement

2018-05-02 Thread Robert Haas
On Tue, May 1, 2018 at 6:37 PM, Peter Geoghegan wrote: > This seems to be an old idea. I'm not too surprised ... this area has been well-studied. > I've always had a problem with the 8GB/16GB upper limit on the size of > shared_buffers. Not because it's wrong, but because it's not something > th

Re: doc fixes: vacuum_cleanup_index_scale_factor

2018-05-02 Thread Alexander Korotkov
Hi! Thank you for your attention on this subject. It's definitely right, that documentation needs to be revised in these places. On Wed, May 2, 2018 at 6:43 PM, Justin Pryzby wrote: > index eabe2a9..e305de9 100644 > --- a/doc/src/sgml/config.sgml > +++ b/doc/src/sgml/config.sgml > @@ -1893,14

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Peter Geoghegan
On Wed, May 2, 2018 at 8:46 AM, Robert Haas wrote: > On Wed, May 2, 2018 at 11:38 AM, Heikki Linnakangas wrote: >> To fix, I propose that we change the above so that we always subtract >> tapeSpace, but if there is less than e.g. 32 kB of memory left after that >> (including, if it went below 0),

Re: Is there a memory leak in commit 8561e48?

2018-05-02 Thread Peter Eisentraut
On 5/1/18 11:42, Tom Lane wrote: > Peter Eisentraut writes: >> The memory leak can be fixed by adding a pfree(). > > That seems like an odd way to approach this. Why not just remove the > reset of _SPI_stack and _SPI_stack_depth, so as to subtract code rather > than adding it --- that is, make i

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-05-02 Thread Peter Eisentraut
On 5/2/18 02:26, Amit Langote wrote: > You're right. I think that's what you were also saying on the other > thread, in reply to which I directed you to this thread. I very clearly > missed that BEFORE ROW triggers are still disallowed. fixed -- Peter Eisentraut http://www.2ndQuad

Re: stats_ext test fails with -DCATCACHE_FORCE_RELEASE

2018-05-02 Thread Tom Lane
Amit Langote writes: > On 2018/05/02 0:33, Tom Lane wrote: >> Amit Langote writes: >>> While playing around with a -DCATCACHE_FORCE_RELEASE build, I noticed that >>> stats_ext test failed with errors for multiple statements that looked like >>> this: >>> ERROR: invalid ndistinct magic 7f7f7f7f (

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Robert Haas
On Wed, May 2, 2018 at 11:38 AM, Heikki Linnakangas wrote: > To fix, I propose that we change the above so that we always subtract > tapeSpace, but if there is less than e.g. 32 kB of memory left after that > (including, if it went below 0), then we bump availMem back up to 32 kB. So > we'd always

Re: doc fixes: vacuum_cleanup_index_scale_factor

2018-05-02 Thread Robert Haas
On Wed, May 2, 2018 at 11:43 AM, Justin Pryzby wrote: > 2nd attempt That looks like good English to me. I can't vouch for whether it's technically correct. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Robert Haas
On Tue, May 1, 2018 at 8:12 PM, Yuriy Zhuravlev wrote: >> That indeed would be an improvement, but maybe we could fix that specific >> pain point without having to throw away twenty years worth of work? > > Indeed! Only a few thousands of lines of code can generate the whole you > manually wrote,

Re: doc fixes: vacuum_cleanup_index_scale_factor

2018-05-02 Thread Justin Pryzby
On Wed, May 02, 2018 at 10:54:31AM -0400, Robert Haas wrote: > On Tue, May 1, 2018 at 10:30 PM, Justin Pryzby wrote: > > -When no tuples were deleted from the heap, B-tree indexes might > > still > > -be scanned during VACUUM cleanup stage by two > > -reasons. The first r

Sort performance cliff with small work_mem

2018-05-02 Thread Heikki Linnakangas
Hi, I spent some time performance testing sorting, and spotted a funny phenomenon with very small work_mem settings. This example demonstrates it well: I sorted about 1 GB worth of pre-sorted integers, with different settings of work_mem, between 64 kB (the minimum), and 100 kB. I also adde

Re: GSoC 2018: thrift encoding format

2018-05-02 Thread Aleksander Alekseev
Hello Charles, > Can I assume the data in thrift is always send inside a struct? Sure! > I think this question also valid for protobuf? Right, pg_protobuf assumes that data is always a message which is an equivalent of Thrift's struct. -- Best regards, Aleksander Alekseev signature.asc Desc

Re: Explain buffers wrong counter with parallel plans

2018-05-02 Thread Adrien Nayrat
Hi, On 05/02/2018 05:22 PM, Robert Haas wrote: > On Tue, May 1, 2018 at 4:57 PM, Andres Freund wrote: >> Robert, you added this as an open item. I don't think it's clear that >> there's a bug here, and even less clear that it's something new for >> v11. Am I understanding that right? Should we c

Re: Explain buffers wrong counter with parallel plans

2018-05-02 Thread Robert Haas
On Tue, May 1, 2018 at 4:57 PM, Andres Freund wrote: > Robert, you added this as an open item. I don't think it's clear that > there's a bug here, and even less clear that it's something new for > v11. Am I understanding that right? Should we close this open item? Yeah, I added it in response to

Re: doc fixes: vacuum_cleanup_index_scale_factor

2018-05-02 Thread Robert Haas
On Tue, May 1, 2018 at 10:30 PM, Justin Pryzby wrote: > -When no tuples were deleted from the heap, B-tree indexes might still > -be scanned during VACUUM cleanup stage by two > -reasons. The first reason is that B-tree index contains deleted > pages > -which can

Re: lingering references to V0 calling convention

2018-05-02 Thread Heikki Linnakangas
On 23/04/18 14:17, John Naylor wrote: I noticed one in the SPI docs and tried to look for more. The attached patch is an attempt at removing the remaining references. The original SPI example returned a uint64 as a signed int8 SQL type, so I'm not sure if I handled that correctly. Committed, th

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Hartmut Holzgraefe
On 02.05.2018 16:22, Tom Lane wrote: (-D? really?) it's worse ... "cmake -L" only produces a alphabetically sorted list of known -D settings, just listing the names without description. That's so far behind from what ./configure --help produces. (And don't get me started on cmake-gui. On

Re: power() function in Windows: "value out of range: underflow"

2018-05-02 Thread Tom Lane
"David G. Johnston" writes: > On Tue, May 1, 2018 at 12:08 PM, Robert Haas wrote: >> That's probably true, but making dev, test, and production boxes >> similar is generally good practice and users can do as much or as >> little of it as they find they need in order to avoid getting burned. >> Th

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Tom Lane
Thomas Munro writes: > Ahh, OpenSSL's armcap.c shows how to do this. You need to > siglongjmp() out of there. Here's a patch that does it that way. > Isn't this better? Do you really need the pg_crc32c_armv8_choose_dummy global variable? That seems pretty ugly. If you're concerned about the co

Re: BufFileSize() doesn't work on a "shared" BufFiles

2018-05-02 Thread Heikki Linnakangas
On 30/04/18 22:00, Peter Geoghegan wrote: On Mon, Apr 30, 2018 at 10:18 AM, Heikki Linnakangas wrote: Perhaps that would be OK, if it was properly commented. But it's not actually hard to make BufFileSize() work on shared files, too, so I think we should do that. I agree that this could use a

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Tom Lane
Geoff Winkless writes: > Being blunt, unless I've missed the point all the arguments I've read so > far for cmake seem to be advantages for the developers, not the users. As > developers who put in your time you are of course entitled to make your > lives easier but I'm just making the counterpoin

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread David G. Johnston
On Wed, May 2, 2018 at 1:07 AM, Amit Langote wrote: > Hi David. > > On 2018/05/02 8:18, David Rowley wrote: > > On 1 May 2018 at 21:44, Amit Langote > wrote: > > > > I re-read the patch and it still looks fine to me. I'm sure it could > > be made better, but I just don't currently see how. I thi

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Apr 24, 2018 at 5:59 PM, Alvaro Herrera > wrote: > > Constraint > > exclusion was pretty easy to get wrong, hence the need for a separate > > section, and I suppose the new partition pruning may be prey to the same > > problems, so it seems worth to document them spec

Re: Remove mention in docs that foreign keys on partitioned tables are not supported

2018-05-02 Thread Ashutosh Bapat
On Wed, May 2, 2018 at 11:56 AM, Amit Langote wrote: > But one could very well argue that BEFORE ROW triggers on the > parent should run before performing the tuple routing and not be cloned to > individual partitions, in which case the result would not have been the > same. Perhaps that's the mo

Re: inconsistency and inefficiency in setup_conversion()

2018-05-02 Thread John Naylor
On 4/28/18, Tom Lane wrote: > John Naylor writes: >> Solution #1 - As alluded to in [1], turn the conversions into >> pg_proc.dat and pg_conversion.dat entries. Teach genbki.pl to parse >> pg_wchar.h to map conversion names to numbers. >> Pros: >> -likely easy to do >> -allows for the removal of

Re: FPW stats?

2018-05-02 Thread Dmitry Dolgov
> On 2 May 2018 at 13:10, Michael Paquier wrote: > On Wed, May 02, 2018 at 12:22:34PM +0200, Dmitry Dolgov wrote: >> Recently I've heard people complaining that Postgres doesn't expose any >> statistics about how many full page writes happened during some time >> frame. > > pg_waldump --stats? Ye

Re: Optimize Arm64 crc32c implementation in Postgresql

2018-05-02 Thread Thomas Munro
On Thu, Apr 5, 2018 at 12:00 AM, Thomas Munro wrote: > ... trying to figure out how to detect the instruction portably using SIGILL > ... Ahh, OpenSSL's armcap.c shows how to do this. You need to siglongjmp() out of there. Here's a patch that does it that way. Isn't this better? I tested this

Re: FPW stats?

2018-05-02 Thread Michael Paquier
On Wed, May 02, 2018 at 12:22:34PM +0200, Dmitry Dolgov wrote: > Recently I've heard people complaining that Postgres doesn't expose any > statistics about how many full page writes happened during some time > frame. pg_waldump --stats? > I guess it can be implemented in a more effective and opti

Re: Oddity in tuple routing for foreign partitions

2018-05-02 Thread Etsuro Fujita
(2018/05/02 15:45), Etsuro Fujita wrote: (2018/05/02 10:10), Amit Langote wrote: On 2018/05/02 6:09, Andres Freund wrote: On 2018-05-01 13:41:32 -0400, Robert Haas wrote: Committed. Here is a small patch to remove a no-longer-needed cast in postgresBeginForeignInsert(). Best regards, Etsu

Re: Is a modern build system acceptable for older platforms

2018-05-02 Thread Geoff Winkless
On Wed, 2 May 2018 at 00:57, Yuriy Zhuravlev wrote: > Hello Geoff! > > About cmake: > 1. You can still use the binary build for your system. > 2. You can still build Postgres from source and with old gcc, you need > only install cmake (it's very easy) Only most modern versions of CMake > depend

FPW stats?

2018-05-02 Thread Dmitry Dolgov
Hi, Recently I've heard people complaining that Postgres doesn't expose any statistics about how many full page writes happened during some time frame. Indeed, I couldn't find any easy way to do so, and judging from my understanding of xloginsert.c it actually can be done per database with the att

Re: Should we add GUCs to allow partition pruning to be disabled?

2018-05-02 Thread Amit Langote
Hi David. On 2018/05/02 8:18, David Rowley wrote: > On 1 May 2018 at 21:44, Amit Langote wrote: >> About the patch in general, it seems like the newly added documentation >> talks about "Partition Pruning" as something that *replaces* constraint >> exclusion. But, I think "Partition Pruning" is

Re: Patch missing from back branches

2018-05-02 Thread Haroon
On 1 May 2018 at 21:04, Tom Lane wrote: > I've pushed this patch into the 9.5 branch, so it should be possible to > spin dory up on that branch now. Thanks Tom! Thanks everyone else for your input on the issue! Regards, Haroon -- Haroonhttp://www.2ndQuadrant.com/

Re: Global snapshots

2018-05-02 Thread Stas Kelvich
> On 2 May 2018, at 05:58, Peter Eisentraut > wrote: > > On 5/1/18 12:27, Stas Kelvich wrote: >> Clock-SI is described in [5] and here I provide a small overview, which >> supposedly should be enough to catch the idea. Assume that each node runs >> Commit >> Sequence Number (CSN) based visibi

Re: Explain buffers wrong counter with parallel plans

2018-05-02 Thread Adrien Nayrat
On 05/01/2018 10:57 PM, Andres Freund wrote: > Robert, you added this as an open item. I don't think it's clear that > there's a bug here, and even less clear that it's something new for > v11. Am I understanding that right? Should we close this open item? Hi, FIY this change is related to 01edb

Re: [HACKERS] Clock with Adaptive Replacement

2018-05-02 Thread Yura Sokolov
02.05.2018 01:37, Peter Geoghegan пишет: > On Tue, May 1, 2018 at 11:58 AM, Robert Haas wrote: >> I agree that double-counting correlated accesses is a a problem, and I >> agree that we probably want to do something about it. I am skeptical >> that using wall-clock time is the right way to solve