Re: [HACKERS] between not propated into a simple equality join

2016-05-09 Thread David Rowley
On 10 May 2016 at 16:34, David G. Johnston wrote: > On Mon, May 9, 2016 at 8:53 AM, Benedikt Grundmann > wrote: >> >> We just run into a very simple query that the planner does much worse on >> than we thought it would (in production the table in question is ~ 100 GB). >> It surprised us given th

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread Pavel Stehule
2016-05-10 8:05 GMT+02:00 David Rowley : > On 10 May 2016 at 16:34, Greg Stark wrote: > > > > On 9 May 2016 8:34 pm, "David Rowley" > wrote: > >> > >> This project does appear to require that we bloat the code with 100's > >> of vector versions of each function. I'm not quite sure if there's a >

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread David Rowley
On 10 May 2016 at 16:34, Greg Stark wrote: > > On 9 May 2016 8:34 pm, "David Rowley" wrote: >> >> This project does appear to require that we bloat the code with 100's >> of vector versions of each function. I'm not quite sure if there's a >> better way to handle this. The problem is that the fmg

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of David Rowley > Sent: Tuesday, May 10, 2016 2:01 PM > To: Kaigai Kouhei(海外 浩平) > Cc: Robert Haas; pgsql-hackers@postgresql.org > Subject: Re: [HACKERS] asynchronous and

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread David Rowley
On 10 May 2016 at 13:38, Kouhei Kaigai wrote: > My concern about ExecProcNode is, it is constructed with a large switch > ... case statement. It involves tons of comparison operation at run-time. > If we replace this switch ... case by function pointer, probably, it make > performance improvement.

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread Greg Stark
On 9 May 2016 8:34 pm, "David Rowley" wrote: > > This project does appear to require that we bloat the code with 100's > of vector versions of each function. I'm not quite sure if there's a > better way to handle this. The problem is that the fmgr is pretty much > a barrier to SIMD operations, and

Re: [HACKERS] between not propated into a simple equality join

2016-05-09 Thread David G. Johnston
On Mon, May 9, 2016 at 8:53 AM, Benedikt Grundmann < bgrundm...@janestreet.com> wrote: > We just run into a very simple query that the planner does much worse on > than we thought it would (in production the table in question is ~ 100 > GB). It surprised us given the planner is generally quite go

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618

2016-05-09 Thread Michael Paquier
On Mon, May 9, 2016 at 4:17 PM, Michael Paquier wrote: > On Tue, Mar 22, 2016 at 1:56 PM, Amit Kapila wrote: >> So as far as I can see there are two ways to resolve this issue, one is to >> retry generation of dsm name if CreateFileMapping returns EACCES and second >> is to append data_dir name t

Re: [HACKERS] Stopping logical replication protocol

2016-05-09 Thread Craig Ringer
On 10 May 2016 at 09:50, Craig Ringer wrote: > I outlined how I think WalSndWaitForWal() should be handled earlier. Test > streamingDoneReceiving > and streamingDoneSending in logical_read_xlog_page and return -1. > OK, so thinking about this some more, I see why you've added the callback with

Re: [HACKERS] Stopping logical replication protocol

2016-05-09 Thread Craig Ringer
> > ProcessRepliesIfAny also now executes in WalSdnWriteData. Because during > send data we should also check message from client(client can send > CopyDone, KeepAlive, Terminate). > > Ah, I didn't spot that ProcessRepliesIfAny() is already called from WalSndWriteData in the current codebase. I ag

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread Kouhei Kaigai
> -Original Message- > From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Robert Haas > Sent: Tuesday, May 10, 2016 2:34 AM > To: pgsql-hackers@postgresql.org > Subject: [HACKERS] asynchronous and vectorized execution > > Hi, > > I realize

[HACKERS] Please help update the "How to beta Test" page

2016-05-09 Thread Josh berkus
Folks, Please help update the wiki page around how to beta test. Particularly, please update it with particular things we'd like to see users test for, like data corruption related to freezing (with some notes on how to test for this). https://wiki.postgresql.org/wiki/HowToBetaTest Thanks! --

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread David Rowley
On 10 May 2016 at 05:33, Robert Haas wrote: > 2. vectorized execution, by which I mean the ability of a node to > return tuples in batches rather than one by one. Andres has opined > more than once that repeated trips through ExecProcNode defeat the > ability of the CPU to do branch prediction co

Re: [HACKERS] Reviewing freeze map code

2016-05-09 Thread Ants Aasma
On Mon, May 9, 2016 at 10:53 PM, Robert Haas wrote: > On Sun, May 8, 2016 at 10:42 PM, Masahiko Sawada > wrote: >> Attached draft patch adds SCANALL option to VACUUM in order to scan >> all pages forcibly while ignoring visibility map information. >> The option name is SCANALL for now but we cou

Re: [HACKERS] Reviewing freeze map code

2016-05-09 Thread Robert Haas
On Sun, May 8, 2016 at 10:42 PM, Masahiko Sawada wrote: > Attached draft patch adds SCANALL option to VACUUM in order to scan > all pages forcibly while ignoring visibility map information. > The option name is SCANALL for now but we could change it after got consensus. If we're going to go that

[HACKERS] GIN logging GIN_SEGMENT_UNMODIFIED actions?

2016-05-09 Thread Andres Freund
Hi, trying to debug something I saw the following in pg_xlogdump output: rmgr: Gin len (rec/tot): 0/ 274, tx: 0, lsn: 1C/DF28AEB0, prev 1C/DF289858, desc: VACUUM_DATA_LEAF_PAGE 3 segments: 5 unknown action 0 ???, blkref #0: rel 1663/16384/16435 blk 310982 note the "seg

Re: [HACKERS] asynchronous and vectorized execution

2016-05-09 Thread Simon Riggs
On 9 May 2016 at 19:33, Robert Haas wrote: > I believe there are other people thinking about these > topics as well, including Andres Freund, Kyotaro Horiguchi, and > probably some folks at 2ndQuadrant (but I don't know exactly who). > 1. asynchronous execution > Not looking at that. > 2.

Re: [HACKERS] Change error code for hstore syntax error

2016-05-09 Thread Sherrylyn Branchaw
Finally returning to this... > Hm, class 42 is generally meant for SQL-level syntax errors. Are these > errors not coming from subroutines of hstore_in()? I think our usual > convention is to use ERRCODE_INVALID_TEXT_REPRESENTATION for complaints > that a data value does not meet its type's con

Re: [HACKERS] Patch for German translation

2016-05-09 Thread Peter Eisentraut
On 5/9/16 10:25 AM, Christian Ullrich wrote: here is a patch for the German translation that removes (all) five instances of *the* most annoying mistake ever. fixed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Service

Re: [HACKERS] [COMMITTERS] pgsql: Add TAP tests for pg_dump

2016-05-09 Thread Stephen Frost
Catalin, * Catalin Iacob (iacobcata...@gmail.com) wrote: > On 5/9/16, Stephen Frost wrote: > >> And what if the tests are buggy? New test suites should go through a > >> CF first I think for proper review. And this is clearly one. > > > > They still won't result in data loss, corruption, or other

Re: [HACKERS] parallel.c is not marked as test covered

2016-05-09 Thread Robert Haas
On Mon, May 9, 2016 at 11:11 AM, Tom Lane wrote: > Andres Freund writes: >> I think it's a good idea to run a force-parallel run on some buildfarm >> members. > > Noah's already doing that on at least one of his critters. But some more > wouldn't hurt. I agree. >> But I'm rather convinced that

[HACKERS] between not propated into a simple equality join

2016-05-09 Thread Benedikt Grundmann
We just run into a very simple query that the planner does much worse on than we thought it would (in production the table in question is ~ 100 GB). It surprised us given the planner is generally quite good, so I thought I share our surprise Setup: postgres_prod@proddb_testing=# select version()

Re: [HACKERS] parallel.c is not marked as test covered

2016-05-09 Thread Tom Lane
Andres Freund writes: > I think it's a good idea to run a force-parallel run on some buildfarm > members. Noah's already doing that on at least one of his critters. But some more wouldn't hurt. > But I'm rather convinced that the core tests run by all animals > need some minimal coverage of par

Re: [HACKERS] [COMMITTERS] pgsql: Add TAP tests for pg_dump

2016-05-09 Thread Catalin Iacob
On 5/9/16, Stephen Frost wrote: >> And what if the tests are buggy? New test suites should go through a >> CF first I think for proper review. And this is clearly one. > > They still won't result in data loss, corruption, or other direct impact > on our users, even if they are buggy. They also wo

Re: [HACKERS] parallel.c is not marked as test covered

2016-05-09 Thread Andres Freund
On 2016-05-08 22:20:55 -0300, Alvaro Herrera wrote: > David Rowley wrote: > > > I'm not entirely sure which machine generates that coverage output, > > but the problem with it is that it's just one machine. We do have at > > least one buildfarm member which runs with force_parallel_mode = > > regr

Re: [HACKERS] Patch for German translation

2016-05-09 Thread Euler Taveira
On 09-05-2016 11:25, Christian Ullrich wrote: > here is a patch for the German translation that removes (all) five > instances of *the* most annoying mistake ever. > We generally don't work with patches for translations. Instead, we send all PO files. We have a separate repository [1] to deal with

Re: [HACKERS] Patch for German translation

2016-05-09 Thread Tom Lane
Christian Ullrich writes: > By the way, is there any documentation anywhere about how and where to > send translation patches? I'm just guessing here. https://wiki.postgresql.org/wiki/NLS regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgres

[HACKERS] Patch for German translation

2016-05-09 Thread Christian Ullrich
Hello, here is a patch for the German translation that removes (all) five instances of *the* most annoying mistake ever. By the way, is there any documentation anywhere about how and where to send translation patches? I'm just guessing here. -- Christian diff --git a/de/postgres.po b/de/po

Re: [HACKERS] [COMMITTERS] pgsql: Add TAP tests for pg_dump

2016-05-09 Thread Stephen Frost
* Michael Paquier (michael.paqu...@gmail.com) wrote: > On Sun, May 8, 2016 at 6:44 AM, Stephen Frost wrote: > > I do think that now is a good time for people to be reviewing what's > > been committed, which includes writing tests (either automated ones, > > which can be included in our test suite,

Re: [HACKERS] force_parallel_mode uniqueness

2016-05-09 Thread Robert Haas
On Sun, May 8, 2016 at 2:23 PM, David G. Johnston wrote: > I was thinking more along the lines that it should be called: > > parallel_mode (enum) > > It would default to "on" > > "off" would turn it off (instead of having to set parallel_degree to 0) > > And it would have additional enum values fo

Re: [HACKERS] A population of population counts

2016-05-09 Thread Robert Haas
On Sun, May 8, 2016 at 7:46 PM, Thomas Munro wrote: > My aim with this thread was mainly reducing code duplication and > needless code: perhaps at least the other ideas in the attached > sketch, namely using ffs instead of the rightmost_one_pos table loop > and consolidation of popcount into a reu

Re: [HACKERS] Declarative partitioning

2016-05-09 Thread Ashutosh Bapat
Hi Amit, I am trying multi-column/expression partitions. create table t1_multi_col (a int, b int) partition by range (a, b); create table t1_mc_p1 partition of t1_multi_col for values start (1, 200) end (100, 300); create table t1_mc_p2 partition of t1_multi_col for values start (200, 1) end (300,

Re: [HACKERS] Stopping logical replication protocol

2016-05-09 Thread Vladimir Gordiychuk
> > What's your PostgreSQL community username? gordiychuk It seems like what you're also trying to allow interruption deeper than > that, when we're in the middle of processing a reorder buffer commit record > and streaming that to the client. You're introducing an is_active member > (actually a

[HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Re: [HACKERS] Windows service is not starting so there’s message in log: FATAL: "could not create shared memory segment “Global/PostgreSQL.851401618

2016-05-09 Thread Michael Paquier
On Tue, Mar 22, 2016 at 1:56 PM, Amit Kapila wrote: > So as far as I can see there are two ways to resolve this issue, one is to > retry generation of dsm name if CreateFileMapping returns EACCES and second > is to append data_dir name to dsm name as the same is done for main shared > memory, that

Re: [HACKERS] pg_shmem_allocations view

2016-05-09 Thread Michael Paquier
On Fri, May 6, 2016 at 8:01 AM, Andres Freund wrote: > Here's a rebased version. I remember why I didn't call the column > "offset" (as Michael complained about upthread), it's a keyword... Oh, an old patch resurrected from the dead... Reading again the patch +* Increase number of initilize