Re: POC: converting Lists into arrays

2019-08-07 Thread Craig Ringer
On Thu, 8 Aug 2019 at 12:18, Andres Freund wrote: > Hi, > > On 2019-08-08 11:36:44 +0800, Craig Ringer wrote: > > > you can only put one into the first element of a > > > for (;;). > > > > > > > Use an anonymous block outer scope? Or if not permitted even by C99 > (which > > I think it is), a do

Re: Problem with default partition pruning

2019-08-07 Thread Amit Langote
Hi Alvaro, On Thu, Aug 8, 2019 at 5:27 AM Alvaro Herrera wrote: > On 2019-Aug-07, Simon Riggs wrote: > > I saw your recent commit and it scares me in various places, noted below. > > > > "Commit: Apply constraint exclusion more generally in partitioning" > > > > "This applies particularly to the

Re: POC: converting Lists into arrays

2019-08-07 Thread Andres Freund
Hi, On 2019-08-08 11:36:44 +0800, Craig Ringer wrote: > > you can only put one into the first element of a > > for (;;). > > > > Use an anonymous block outer scope? Or if not permitted even by C99 (which > I think it is), a do {...} while (0); hack? You can't easily - the problem is that there

Re: SegFault on 9.6.14

2019-08-07 Thread Amit Kapila
On Wed, Aug 7, 2019 at 3:15 PM vignesh C wrote: > > On Wed, Jul 31, 2019 at 9:37 AM Amit Kapila wrote: > > > > On Wed, Jul 31, 2019 at 12:05 AM Robert Haas wrote: > > > > The other idea we had discussed which comes closer to adopting Tom's > > position was that during ExecShutdownNode, we just d

Re: Proposal for Signal Detection Refactoring

2019-08-07 Thread Craig Ringer
On Wed, 6 Mar 2019 at 17:38, Chris Travers wrote: > > Here's a new patch. No rush on it. I am moving it to next commitfest > anyway because as code documentation I think this is a low priority late in > the release cycle. > While you're looking at signal detection changes I suggest making sure

Re: POC: converting Lists into arrays

2019-08-07 Thread Craig Ringer
On Thu, 1 Aug 2019 at 07:40, Tom Lane wrote: > Andres Freund writes: > > Unfortunately foreach(ListCell *lc, ...) doesn't work with the current > > definition. Which I think isn't great, because the large scopes for loop > > iteration variables imo makes the code harder to reason about. > > Tota

Re: Small patch to fix build on Windows

2019-08-07 Thread Kyotaro Horiguchi
Hello. At Wed, 7 Aug 2019 12:14:48 +0300, Dmitry Igrishin wrote in > > -if ($lib =~ m/\s/) > > -{ > > -$lib = '"' . $lib . """; > > -} > > +# Since VC automatically quotes paths specified as the data of > > +# in VC project file, it's mistakably > > +# to quote

Re: Built-in connection pooler

2019-08-07 Thread Ryan Lambert
> I attached to this mail patch which is fixing both problems: correctly > reports error to the client and calculates number of idle clients). > Yes, this works much better with max_sessions=1000. Now it's handling the 300 connections on the small server. n_idle_clients now looks accurate with

Re: Global temporary tables

2019-08-07 Thread Craig Ringer
On Tue, 6 Aug 2019 at 16:32, Konstantin Knizhnik wrote: > New version of the patch with several fixes is attached. > Many thanks to Roman Zharkov for testing. > FWIW I still don't understand your argument with regards to using shared_buffers for temp tables having connection pooling benefits. Ar

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Justin Pryzby
On Wed, Aug 07, 2019 at 04:51:54PM -0700, Andres Freund wrote: > Hi, > > On 2019-08-07 14:29:28 -0500, Justin Pryzby wrote: > > Just found this, although I'm not sure what to do about it. If it's corrupt > > table data, I can restore from backup. In the meantime, I've renamed+uninherited the tab

Re: partition routing layering in nodeModifyTable.c

2019-08-07 Thread Amit Langote
Fujita-san, On Wed, Aug 7, 2019 at 6:00 PM Etsuro Fujita wrote: > On Wed, Aug 7, 2019 at 4:28 PM Amit Langote wrote: > > I just noticed obsolete references to es_result_relation_info that > > 0002 failed to remove. One of them is in fdwhandler.sgml: > > > > > > TupleTableSlot * > > IterateDire

Re: no default hash partition

2019-08-07 Thread Amit Langote
On Thu, Aug 8, 2019 at 6:22 AM Tom Lane wrote: > Alvaro Herrera writes: > > On 2019-Aug-07, Tom Lane wrote: > >> Hm, that's rather confusingly worded IMO. Is the antecedent of "this > >> option" just DEFAULT, or does it mean that you can't use FOR VALUES, > >> or perchance it means that you can'

Re: SQL/JSON path: collation for comparisons, minor typos in docs

2019-08-07 Thread Alexander Korotkov
On Thu, Aug 8, 2019 at 3:05 AM Alexander Korotkov wrote: > On Thu, Aug 8, 2019 at 12:55 AM Alexander Korotkov > wrote: > > On Wed, Aug 7, 2019 at 4:11 PM Alexander Korotkov > > wrote: > > > On Wed, Aug 7, 2019 at 2:25 PM Markus Winand > > > wrote: > > > > I was playing around with JSON path qu

Re: Rethinking opclass member checks and dependency strength

2019-08-07 Thread Alexander Korotkov
On Wed, Aug 7, 2019 at 7:28 PM Tom Lane wrote: > Over in [1] we realized that it would be a good idea to remove the <@ > operator from contrib/intarray's GiST opclasses. Unfortunately, doing > that isn't a simple matter of generating an extension update script > containing ALTER OPERATOR FAMILY D

Re: SQL/JSON path: collation for comparisons, minor typos in docs

2019-08-07 Thread Alexander Korotkov
On Thu, Aug 8, 2019 at 12:55 AM Alexander Korotkov wrote: > On Wed, Aug 7, 2019 at 4:11 PM Alexander Korotkov > wrote: > > On Wed, Aug 7, 2019 at 2:25 PM Markus Winand > > wrote: > > > I was playing around with JSON path quite a bit and might have found one > > > case where the current impleme

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Andres Freund
Hi, On 2019-08-07 14:29:28 -0500, Justin Pryzby wrote: > Just found this, although I'm not sure what to do about it. If it's corrupt > table data, I can restore from backup. > > ts=# VACUUM FREEZE VERBOSE child.huawei_umts_ucell_201908; > INFO: 0: aggressively vacuuming "child.huawei_umts_u

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Sehrope Sarkuni
On Wed, Aug 7, 2019 at 1:39 PM Bruce Momjian wrote: > On Wed, Aug 7, 2019 at 11:41:51AM -0400, Sehrope Sarkuni wrote: > > On Wed, Aug 7, 2019 at 7:19 AM Bruce Momjian wrote: > > > > On Wed, Aug 7, 2019 at 05:13:31PM +0900, Masahiko Sawada wrote: > > > I understood. IIUC in your approac

Re: Documentation clarification re: ANALYZE

2019-08-07 Thread David G. Johnston
On Wed, Aug 7, 2019 at 2:42 PM Isaac Morland wrote: > Thanks. So presumably I would also have permission if I have SET ROLEd to > the owner, or to a role which is an INHERIT member of the owner. > Yes, the table ownership role check walks up the role membership hierarchy if "inherit" is on for t

Re: SQL/JSON path: collation for comparisons, minor typos in docs

2019-08-07 Thread Alexander Korotkov
On Wed, Aug 7, 2019 at 4:11 PM Alexander Korotkov wrote: > On Wed, Aug 7, 2019 at 2:25 PM Markus Winand wrote: > > I was playing around with JSON path quite a bit and might have found one > > case where the current implementation doesn’t follow the standard. > > > > The functionality in question

Re: Documentation clarification re: ANALYZE

2019-08-07 Thread Tom Lane
"David G. Johnston" writes: > On Wed, Aug 7, 2019 at 2:14 PM Isaac Morland > wrote: >> I'm looking at https://www.postgresql.org/docs/current/sql-analyze.html, >> where it says “Without a table_and_columns list, ANALYZE processes every >> table and materialized view in the current database that t

Re: Unused header file inclusion

2019-08-07 Thread Thomas Munro
On Thu, Aug 8, 2019 at 9:47 AM Thomas Munro wrote: > transaction unit *translation unit -- Thomas Munro https://enterprisedb.com

Re: Unused header file inclusion

2019-08-07 Thread Thomas Munro
On Thu, Aug 8, 2019 at 9:00 AM Alvaro Herrera wrote: > On 2019-Aug-05, Tom Lane wrote: > > Alvaro Herrera writes: > > > Then there's the removal, which is in tuplesort.c because of > > > INT_MAX as added by commit d26559dbf356 and still present ... > > > > One has to be especially wary of removi

Re: Documentation clarification re: ANALYZE

2019-08-07 Thread Isaac Morland
On Wed, 7 Aug 2019 at 17:31, David G. Johnston wrote: > On Wed, Aug 7, 2019 at 2:14 PM Isaac Morland > wrote: > >> I'm looking at https://www.postgresql.org/docs/current/sql-analyze.html, >> where it says “Without a table_and_columns list, ANALYZE processes every >> table and materialized view i

Re: Documentation clarification re: ANALYZE

2019-08-07 Thread David G. Johnston
On Wed, Aug 7, 2019 at 2:14 PM Isaac Morland wrote: > I'm looking at https://www.postgresql.org/docs/current/sql-analyze.html, > where it says “Without a table_and_columns list, ANALYZE processes every > table and materialized view in the current database that the current user > has permission to

Re: no default hash partition

2019-08-07 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Aug-07, Tom Lane wrote: >> Hm, that's rather confusingly worded IMO. Is the antecedent of "this >> option" just DEFAULT, or does it mean that you can't use FOR VALUES, >> or perchance it means that you can't use a PARTITION OF clause >> at all? > Uh, you're right

Documentation clarification re: ANALYZE

2019-08-07 Thread Isaac Morland
I'm looking at https://www.postgresql.org/docs/current/sql-analyze.html, where it says “Without a table_and_columns list, ANALYZE processes every table and materialized view in the current database that the current user has permission to analyze.”. I don’t believe there is a separate “analyze” per

Re: no default hash partition

2019-08-07 Thread Tom Lane
Alvaro Herrera writes: > Actually, it also says this (in the blurb for the PARTITION OF clause): > Creates the table as a partition of the specified > parent table. The table can be created either as a partition for > specific > values using FOR VALUES or as a default partition

Re: Avoid full GIN index scan when possible

2019-08-07 Thread Tom Lane
Nikita Glukhov writes: > Attached 6th version of the patches. I spent a bit of time looking at these. Attached is a proposed revision of the 0001 patch, with some minor changes: * I didn't adopt your move of the "Non-default modes require the index to have placeholders" test to after the stanza

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Aug-07, Tom Lane wrote: >> The problem in "timeouts" is that it has to use drearily long timeouts >> to be sure that the behavior will be stable even on really slow machines >> (think CLOBBER_CACHE_ALWAYS or valgrind --- it can take seconds for them >> to reach a w

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Tom Lane
Heikki Linnakangas writes: > The elephant in the room is the 'timeouts' test, which takes about 40 > seconds, out of a total runtime of 90 seconds. So we'd really want to > run that in parallel with everything else. Or split 'timeouts' into > multiple tests that could run in parallel. I don't t

Re: Duplicated LSN in ReorderBuffer

2019-08-07 Thread Andres Freund
Hi, On 2019-08-07 16:19:13 -0400, Alvaro Herrera wrote: > On 2019-Jul-26, Andres Freund wrote: > > > 2) We could simply assign the subtransaction to the parent using > >ReorderBufferAssignChild() in SnapBuildProcessNewCid() or it's > >caller. That ought to also fix the bug > > > >I a

Re: Unused header file inclusion

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-05, Tom Lane wrote: > Alvaro Herrera writes: > > Then there's the removal, which is in tuplesort.c because of > > INT_MAX as added by commit d26559dbf356 and still present ... > > One has to be especially wary of removing system-header inclusions; > the fact that they don't seem to

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Tom Lane
Pavel Stehule writes: > st 7. 8. 2019 v 17:39 odesílatel Tom Lane napsal: >> I wouldn't trust that. You don't really know what the lifespan of >> a fn_extra cache is. > fn_extra cache cannot be longer than query. There are fn_extra caches that are not tied to queries. Admittedly they're for s

Re: Adding a test for speculative insert abort case

2019-08-07 Thread Melanie Plageman
On Wed, Jul 24, 2019 at 11:48 AM Andres Freund wrote: > > diff --git a/src/test/isolation/specs/insert-conflict-specconflict.spec > b/src/test/isolation/specs/insert-conflict-specconflict.spec > > index 3a70484fc2..7f29fb9d02 100644 > > --- a/src/test/isolation/specs/insert-conflict-specconflict.

Re: Problem with default partition pruning

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Simon Riggs wrote: > I saw your recent commit and it scares me in various places, noted below. > > "Commit: Apply constraint exclusion more generally in partitioning" > > "This applies particularly to the default partition..." > > My understanding of the thread was the complaint

Re: Duplicated LSN in ReorderBuffer

2019-08-07 Thread Alvaro Herrera
On 2019-Jul-26, Andres Freund wrote: > 2) We could simply assign the subtransaction to the parent using >ReorderBufferAssignChild() in SnapBuildProcessNewCid() or it's >caller. That ought to also fix the bug > >I also has the advantage that we can save some memory in transactions >

Re: initdb: Use varargs macro for PG_CMD_PRINTF

2019-08-07 Thread Ibrar Ahmed
Hi, Patch does not apply, rebased patch on ( 68343b4ad75305391b38f4b42734dc07f2fe7ee2) attached On Wed, Aug 7, 2019 at 7:04 PM Tom Lane wrote: > Peter Eisentraut writes: > > Small patch to simplify some no longer necessary complicated code, using > > varargs macros. > > +1 > >

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Justin Pryzby
Just found this, although I'm not sure what to do about it. If it's corrupt table data, I can restore from backup. ts=# VACUUM FREEZE VERBOSE child.huawei_umts_ucell_201908; INFO: 0: aggressively vacuuming "child.huawei_umts_ucell_201908" LOCATION: lazy_scan_heap, vacuumlazy.c:502 ERROR: X

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-07 Thread Ashwin Agrawal
On Fri, Aug 2, 2019 at 4:56 PM Ashwin Agrawal wrote: > > On Wed, Jul 31, 2019 at 2:06 PM Andres Freund wrote: > >> Looking at the code as of master, we currently have: >> > > Super awesome feedback and insights, thank you! > > - PredicateLockTuple() calls SubTransGetTopmostTransaction() to figur

Re: Problem with default partition pruning

2019-08-07 Thread Simon Riggs
On Tue, 6 Aug 2019 at 23:18, Alvaro Herrera wrote: > On 2019-Aug-06, Alvaro Herrera wrote: > > > Well, if this is really all that duplicative, one thing we could do is > > run this check in get_partprune_steps_internal only if > > constraint_exclusion is a value other than on; we should achieve t

Re: crash 11.5~ (and 11.4)

2019-08-07 Thread Justin Pryzby
I checked this still happens with max_parallel_workers_per_gather=0. Now, I just reproduced using SELECT * FROM that table: (gdb) p thisatt->attrelid $4 = 2015128626 ts=# SELECT 2015128626::regclass; regclass | child.huawei_umts_ucell_201908 (gdb) p thisatt->attnum $1 = 2 (gdb) p attnum # For e

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Bruce Momjian
On Wed, Aug 7, 2019 at 11:41:51AM -0400, Sehrope Sarkuni wrote: > On Wed, Aug 7, 2019 at 7:19 AM Bruce Momjian wrote: > > On Wed, Aug  7, 2019 at 05:13:31PM +0900, Masahiko Sawada wrote: > > I understood. IIUC in your approach postgres processes encrypt WAL > > records when inserting

crash 11.5~

2019-08-07 Thread Justin Pryzby
A daily report crashed repeatedly this morning running pg11.4. I compiled 11.5 and reproduced it there, too, so I'm including backtrace with -O0. I'm trying to dig further into it, but it seems to be crashing under load, but not when I try to narrow down to a single report, which seem to run to co

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Pavel Stehule
st 7. 8. 2019 v 18:39 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > st 7. 8. 2019 v 17:39 odesílatel Tom Lane napsal: > >> I wouldn't trust that. You don't really know what the lifespan of > >> a fn_extra cache is. > > > fn_extra cache cannot be longer than query. > > There are fn_ex

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Tom Lane wrote: > The problem in "timeouts" is that it has to use drearily long timeouts > to be sure that the behavior will be stable even on really slow machines > (think CLOBBER_CACHE_ALWAYS or valgrind --- it can take seconds for them > to reach a waiting state that other machi

Re: no default hash partition

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Tom Lane wrote: > Alvaro Herrera writes: > > Actually, it also says this (in the blurb for the PARTITION OF clause): > > > Creates the table as a partition of the > > specified > > parent table. The table can be created either as a partition for > > specific > >

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Heikki Linnakangas wrote: > The elephant in the room is the 'timeouts' test, which takes about 40 > seconds, out of a total runtime of 90 seconds. So we'd really want to run > that in parallel with everything else. Or split 'timeouts' into multiple > tests that could run in paralle

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 18:52, Alvaro Herrera wrote: On 2019-Aug-07, Tom Lane wrote: Something related I've been wondering about is whether we could parallelize the isolation tests. A difficulty here is that the slowest ones tend to also be timing-sensitive, such that running them in parallel would incr

Re: no default hash partition

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Amit Langote wrote: > That hash-partitioned tables can't have default partition is mentioned > in the CREATE TABLE page: > > "If DEFAULT is specified, the table will be created as a default > partition of the parent table. The parent can either be a list or > range partitioned tab

Rethinking opclass member checks and dependency strength

2019-08-07 Thread Tom Lane
Over in [1] we realized that it would be a good idea to remove the <@ operator from contrib/intarray's GiST opclasses. Unfortunately, doing that isn't a simple matter of generating an extension update script containing ALTER OPERATOR FAMILY DROP OPERATOR, because that operator is marked as interna

Re: no default hash partition

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-06, Stephen Frost wrote: > Yeah, that's a fair argument, but giving the user a way to say that > would address it. As in, "create me a list-partitioned table for these > values, plus a default." Anyhow, I'm sure that I'm taking this beyond > what we need to do right now, just sharing

Re: Removing unneeded self joins

2019-08-07 Thread Konstantin Knizhnik
On 05.08.2019 14:24, Andrey Lepikhov wrote: On 02/08/2019 04:54, Thomas Munro wrote: On Thu, Jun 27, 2019 at 6:42 PM Andrey Lepikhov wrote: Version v.17 of the patch that fix the bug see in attachment. While moving this to the September CF, I noticed that it needs to be updated for the r

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Pavel Stehule
st 7. 8. 2019 v 17:39 odesílatel Tom Lane napsal: > Pavel Stehule writes: > > I should to use a cache accessed via fn_extra. There will be stored data > > about function parameters (types). If I understand correctly, these data > > should be stable in query, and then recheck is not necessary. Is

Re: s/rewinded/rewound/?

2019-08-07 Thread Geoff Winkless
On Wed, 7 Aug 2019 at 16:59, Alvaro Herrera wrote: > He didn't > add a mailing list reference, but this is easy to find at > https://postgr.es/m/20160720180706.gf24...@momjian.us > I lean towards the view that he was using the literal program name as a > verb, rather than trying to decline a verb

Re: stress test for parallel workers

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 17:45, Tom Lane wrote: Heikki Linnakangas writes: On 07/08/2019 16:57, Tom Lane wrote: Also, if you're using systemd or something else that thinks it ought to interfere with where cores get dropped, that could be a problem. I think they should just go to a file called "core",

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Chapman Flack
On 08/07/19 11:39, Tom Lane wrote: > Pavel Stehule writes: >> I should to use a cache accessed via fn_extra. There will be stored data >> about function parameters (types). If I understand correctly, these data >> should be stable in query, and then recheck is not necessary. Is it true? > > I wou

Re: s/rewinded/rewound/?

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Tom Lane wrote: > Magnus Hagander writes: > > To me this sounds like a classic non-English-native-speaker-mistake. But > > it seems at least the one in the docs come from Bruce, who definitely is... > > He might've just been committing somebody else's words without having > rev

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Tom Lane wrote: > Something related I've been wondering about is whether we could > parallelize the isolation tests. A difficulty here is that the > slowest ones tend to also be timing-sensitive, such that running > them in parallel would increase the risk of failure. But we > co

Re: s/rewinded/rewound/?

2019-08-07 Thread Tom Lane
Magnus Hagander writes: > On Wed, Aug 7, 2019 at 10:49 AM Thomas Munro wrote: >> The word "rewinded" appears in our manual and in a comment. That >> sounds strange to my ears. Isn't it a mistake? Certainly. > To me this sounds like a classic non-English-native-speaker-mistake. But > it seems

Re: Handling RestrictInfo in expression_tree_walker

2019-08-07 Thread Tom Lane
Konstantin Knizhnik writes: > Frankly speaking I do not see some good reasons for not handling > RestrictInfo in expression_tree_worker. It can really simplify writing > of mutators/walkers. I don't buy this; what seems more likely is that you're trying to apply an expression tree mutator to so

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Sehrope Sarkuni
On Wed, Aug 7, 2019 at 7:19 AM Bruce Momjian wrote: > On Wed, Aug 7, 2019 at 05:13:31PM +0900, Masahiko Sawada wrote: > > I understood. IIUC in your approach postgres processes encrypt WAL > > records when inserting to the WAL buffer. So WAL data is encrypted > > even on the WAL buffer. > I was

Re: is necessary to recheck cached data in fn_extra?

2019-08-07 Thread Tom Lane
Pavel Stehule writes: > I should to use a cache accessed via fn_extra. There will be stored data > about function parameters (types). If I understand correctly, these data > should be stable in query, and then recheck is not necessary. Is it true? I wouldn't trust that. You don't really know wha

Re: Unix-domain socket support on Windows

2019-08-07 Thread Magnus Hagander
On Wed, Aug 7, 2019 at 4:59 PM Peter Eisentraut < peter.eisentr...@2ndquadrant.com> wrote: > On 2019-08-07 16:06, Heikki Linnakangas wrote: > > Am I reading the patches correctly, that getpeereid() still doesn't work > > on Windows? That means that peer authentication doesn't work, right? > > That

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Tom Lane
Heikki Linnakangas writes: > The list of tests in src/test/isolation/isolation_schedule has grown > over the years. Originally, they were all related to Serializable > Snapshot Isolation, but there are different kinds of concurrency tests > there now. More tests is good, but the schedule file h

Re: no default hash partition

2019-08-07 Thread Alvaro Herrera
On 2019-Aug-07, Amit Langote wrote: > That hash-partitioned tables can't have default partition is mentioned > in the CREATE TABLE page: > > "If DEFAULT is specified, the table will be created as a default > partition of the parent table. The parent can either be a list or > range partitioned tab

Re: Unix-domain socket support on Windows

2019-08-07 Thread Peter Eisentraut
On 2019-08-07 16:06, Heikki Linnakangas wrote: > Am I reading the patches correctly, that getpeereid() still doesn't work > on Windows? That means that peer authentication doesn't work, right? > That's a bit sad. One of the big advantages of unix domain sockets over > TCP is peer authentication.

Re: stress test for parallel workers

2019-08-07 Thread Tom Lane
Heikki Linnakangas writes: > On 07/08/2019 16:57, Tom Lane wrote: >> Also, if you're using systemd or something else that thinks it >> ought to interfere with where cores get dropped, that could be >> a problem. > I think they should just go to a file called "core", I don't think I've > changed

Re: stress test for parallel workers

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 16:57, Tom Lane wrote: Heikki Linnakangas writes: On 07/08/2019 02:57, Thomas Munro wrote: On Wed, Jul 24, 2019 at 5:15 PM Tom Lane wrote: So I think I've got to take back the assertion that we've got some lurking generic problem. This pattern looks way more like a platform-sp

Re: Regression test failure in regression test temp.sql

2019-08-07 Thread Tom Lane
Michael Paquier writes: > While browsing the buildfarm failures, I have found this problem on > anole for the test temp: > ... > This could be solved just with an ORDER BY as per the attached. Any > objections? There's no reason to expect stability of row order in pg_class, so in principle this

Re: Unix-domain socket support on Windows

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 16:56, Peter Eisentraut wrote: It works! Cool! Am I reading the patches correctly, that getpeereid() still doesn't work on Windows? That means that peer authentication doesn't work, right? That's a bit sad. One of the big advantages of unix domain sockets over TCP is peer auth

Re: initdb: Use varargs macro for PG_CMD_PRINTF

2019-08-07 Thread Tom Lane
Peter Eisentraut writes: > Small patch to simplify some no longer necessary complicated code, using > varargs macros. +1 regards, tom lane

Re: no default hash partition

2019-08-07 Thread Stephen Frost
Greetings, * Amit Langote (amitlangot...@gmail.com) wrote: > On Wed, Aug 7, 2019 at 1:59 PM Kyotaro Horiguchi > wrote: > > At Tue, 6 Aug 2019 23:26:19 -0400, Robert Haas > > wrote: > > > On Tue, Aug 6, 2019 at 6:58 PM Tom Lane wrote: > > > I think, as Amit says, that having an automatic partit

Re: stress test for parallel workers

2019-08-07 Thread Tom Lane
Heikki Linnakangas writes: > On 07/08/2019 02:57, Thomas Munro wrote: >> On Wed, Jul 24, 2019 at 5:15 PM Tom Lane wrote: >>> So I think I've got to take back the assertion that we've got >>> some lurking generic problem. This pattern looks way more >>> like a platform-specific issue. Overaggres

Unix-domain socket support on Windows

2019-08-07 Thread Peter Eisentraut
It works! (apparently as of Windows 10 version 1803) Here are some patches to get a discussion rolling. Basically, it just works, but you need to define your own struct sockaddr_un. (This is what configure currently uses as a proxy for HAVE_UNIX_SOCKETS, so (a) that needs a bit of tweaking, and

Re: Small patch to fix build on Windows

2019-08-07 Thread Dmitry Igrishin
ср, 7 авг. 2019 г. в 15:33, Juan José Santamaría Flecha : > > On Wed, Aug 7, 2019 at 11:11 AM Dmitry Igrishin wrote: > > > > ср, 7 авг. 2019 г. в 11:29, Kyotaro Horiguchi : > > > > > > Solution.pm has the following line: > > > > > > > my $opensslcmd = > > > > $self->{options}->{opens

Re: stress test for parallel workers

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 02:57, Thomas Munro wrote: On Wed, Jul 24, 2019 at 5:15 PM Tom Lane wrote: So I think I've got to take back the assertion that we've got some lurking generic problem. This pattern looks way more like a platform-specific issue. Overaggressive OOM killer would fit the facts on vul

Regression test failure in regression test temp.sql

2019-08-07 Thread Michael Paquier
Hi all, While browsing the buildfarm failures, I have found this problem on anole for the test temp: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=anole&dt=2019-08-07%2006%3A39%3A35 select relname from pg_class where relname like 'temp_parted_oncommit_test%'; relname ---

Re: SQL/JSON path: collation for comparisons, minor typos in docs

2019-08-07 Thread Alexander Korotkov
Hi! On Wed, Aug 7, 2019 at 2:25 PM Markus Winand wrote: > I was playing around with JSON path quite a bit and might have found one case > where the current implementation doesn’t follow the standard. > > The functionality in question are the comparison operators except ==. They > use the databa

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Sehrope Sarkuni
On Mon, Aug 5, 2019 at 9:02 PM Bruce Momjian wrote: > On Wed, Jul 31, 2019 at 09:25:01AM -0400, Sehrope Sarkuni wrote: > > Even if we do not include a separate per-relation salt or things like > > relfilenode when generating a derived key, we can still include other > types of > > immutable attri

Re: Built-in connection pooler

2019-08-07 Thread Ryan Lambert
> First of all default value of this parameter is 1000, not 10. Oops, my bad! Sorry about that, I'm not sure how I got that in my head last night but I see how that would make it act strange now. I'll adjust my notes before re-testing. :) Thanks, *Ryan Lambert* On Wed, Aug 7, 2019 at 4:57 A

initdb: Use varargs macro for PG_CMD_PRINTF

2019-08-07 Thread Peter Eisentraut
Small patch to simplify some no longer necessary complicated code, using varargs macros. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services From 5004af6cc9174e9cdc20a6bf4f959d22e209a8e8 Mon Sep 17 00:00:00 2001 From:

Re: Small patch to fix build on Windows

2019-08-07 Thread Juan José Santamaría Flecha
On Wed, Aug 7, 2019 at 11:11 AM Dmitry Igrishin wrote: > > ср, 7 авг. 2019 г. в 11:29, Kyotaro Horiguchi : > > > > Solution.pm has the following line: > > > > > my $opensslcmd = > > > $self->{options}->{openssl} . "\\bin\\openssl.exe version 2>&1"; > > > > AFAICS that's all. > Thank

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 14:42, Thomas Munro wrote: On Wed, Aug 7, 2019 at 11:28 PM Heikki Linnakangas wrote: The list of tests in src/test/isolation/isolation_schedule has grown over the years. Originally, they were all related to Serializable Snapshot Isolation, but there are different kinds of concurre

Re: s/rewinded/rewound/?

2019-08-07 Thread Chapman Flack
On 08/07/19 04:48, Thomas Munro wrote: > as "poetic" and "rare", and then says it was used by one specific > Victorian poet. Perhaps I'll send them a pull request: it's now G. M. > Hopkins and PostgreSQL? It does seem counter, original, spare, strange. Regards, -Chap

Re: Grouping isolationtester tests in the schedule

2019-08-07 Thread Thomas Munro
On Wed, Aug 7, 2019 at 11:28 PM Heikki Linnakangas wrote: > The list of tests in src/test/isolation/isolation_schedule has grown > over the years. Originally, they were all related to Serializable > Snapshot Isolation, but there are different kinds of concurrency tests > there now. More tests is g

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Thomas Munro
On Thu, Aug 1, 2019 at 1:22 AM Amit Kapila wrote: > On Wed, Jul 31, 2019 at 10:13 AM Amit Kapila wrote: > > On Tue, Jul 30, 2019 at 5:26 PM Thomas Munro wrote: > > > but > > > here's a small thing: I managed to reach an LWLock self-deadlock in > > > the undo worker launcher: > > > > > > > I cou

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Heikki Linnakangas
On 07/08/2019 13:52, Dilip Kumar wrote: I have one more problem related to compression of the command id field. Basically, the problem is that we don't set the command id in the WAL and we will always store FirstCommandId in the undo[1]. So suppose there were 2 operations under a different CID

Grouping isolationtester tests in the schedule

2019-08-07 Thread Heikki Linnakangas
The list of tests in src/test/isolation/isolation_schedule has grown over the years. Originally, they were all related to Serializable Snapshot Isolation, but there are different kinds of concurrency tests there now. More tests is good, but the schedule file has grown into a big inscrutable lis

SQL/JSON path: collation for comparisons, minor typos in docs

2019-08-07 Thread Markus Winand
Hi! I was playing around with JSON path quite a bit and might have found one case where the current implementation doesn’t follow the standard. The functionality in question are the comparison operators except ==. They use the database default collation rather then the standard-mandated "Unicod

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-07 Thread Bruce Momjian
On Wed, Aug 7, 2019 at 05:13:31PM +0900, Masahiko Sawada wrote: > I understood. IIUC in your approach postgres processes encrypt WAL > records when inserting to the WAL buffer. So WAL data is encrypted > even on the WAL buffer. > > It works but I think the implementation might be complex; For exa

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Heikki Linnakangas
On 05/08/2019 16:24, Robert Haas wrote: On Sun, Aug 4, 2019 at 5:16 AM Heikki Linnakangas wrote: I feel that the level of abstraction is not quite right. There are a bunch of fields, like uur_block, uur_offset, uur_tuple, that are probably useful for some UNDO resource managers (zheap I presume

Re: Store FullTransactionId in TwoPhaseFileHeader/GlobalTransactionData

2019-08-07 Thread vignesh C
On Mon, Aug 5, 2019 at 8:31 AM Andres Freund wrote: > > Hi, > > On 2019-08-05 14:44:37 +1200, Thomas Munro wrote: > > Yeah. I think we're agreed for now that we don't want to change > > procarray (though we still need to figure out how to compute the 64 > > bit horizons correctly and efficiently)

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Dilip Kumar
On Tue, Jul 30, 2019 at 1:54 PM Dilip Kumar wrote: > > On Tue, Jul 30, 2019 at 12:21 PM Thomas Munro wrote: > > > > One data structure that could perhaps hold this would be > > UndoLogTableEntry (the per-backend cache, indexed by undo log number, > > with pretty fast lookups; used for things like

Re: Remove HeapTuple and Buffer dependency for predicate locking functions

2019-08-07 Thread Heikki Linnakangas
On 06/08/2019 13:35, Thomas Munro wrote: On Tue, Aug 6, 2019 at 9:26 PM Heikki Linnakangas wrote: Attached is a patch that contains your fix.txt with the changes for clarity mentioned above, and an isolationtester test case. LGTM. Pushed, thanks! - Heikki

Re: block-level incremental backup

2019-08-07 Thread Ibrar Ahmed
On Wed, Aug 7, 2019 at 2:47 PM Jeevan Chalke wrote: > > > On Mon, Aug 5, 2019 at 7:13 PM Robert Haas wrote: > >> On Fri, Aug 2, 2019 at 9:13 AM vignesh C wrote: >> > + rc = system(copycmd); >> >> I don't think this patch should be calling system() in the first place. >> > > So, do you mean we s

Re: s/rewinded/rewound/?

2019-08-07 Thread Liudmila Mantrova
On 8/7/19 12:00 PM, Michael Paquier wrote: On Wed, Aug 07, 2019 at 10:53:45AM +0200, Magnus Hagander wrote: To me this sounds like a classic non-English-native-speaker-mistake. But it seems at least the one in the docs come from Bruce, who definitely is... So perhaps it's intentional to refer t

Re: block-level incremental backup

2019-08-07 Thread Jeevan Chalke
On Mon, Aug 5, 2019 at 7:13 PM Robert Haas wrote: > On Fri, Aug 2, 2019 at 9:13 AM vignesh C wrote: > > + rc = system(copycmd); > > I don't think this patch should be calling system() in the first place. > So, do you mean we should just do fread() and fwrite() for the whole file? I thought it

Re: SegFault on 9.6.14

2019-08-07 Thread vignesh C
On Wed, Jul 31, 2019 at 9:37 AM Amit Kapila wrote: > > On Wed, Jul 31, 2019 at 12:05 AM Robert Haas wrote: > > > > On Thu, Jul 18, 2019 at 9:45 AM Tom Lane wrote: > > > I think this is going in the wrong direction. Nodes should *always* > > > assume that a rescan is possible until ExecEndNode i

Re: POC: Cleaning up orphaned files using undo logs

2019-08-07 Thread Amit Kapila
On Tue, Aug 6, 2019 at 1:26 PM Andres Freund wrote: > On 2019-08-05 11:29:34 -0700, Andres Freund wrote: > I am responding to some of the points where I need some more inputs or some discussion is required. Some of the things need more thoughts which I will respond later and some are quite strai

Re: Small patch to fix build on Windows

2019-08-07 Thread Dmitry Igrishin
ср, 7 авг. 2019 г. в 11:29, Kyotaro Horiguchi : > > Hi, > > At Tue, 6 Aug 2019 22:50:14 +0300, Dmitry Igrishin wrote > in > > The attached self-documented patch fixes build on Windows in case when > > path to Python has embedded spaces. > > - $solution->{options}->{python} . "\\python -

  1   2   >