Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-29 Thread Alvaro Herrera
Tom Lane wrote: > My own thought is that there's room for at least a few days' slop in > the end date of the final commitfest, depending on what patches remain > open and what the prospects are for getting them done. (In the past > we've sometimes let the final fest stretch on indefinitely, which

Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-29 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Mar 29, 2017 at 3:04 PM, Alvaro Herrera > wrote: > > I was rather surprised to see the March commitfest declared to exactly > > one month and feature freeze immediately thereafter. > > That's true, but at >

Re: [HACKERS] error handling in RegisterBackgroundWorker

2017-03-29 Thread Alvaro Herrera
Robert Haas wrote: > On Wed, Mar 29, 2017 at 2:10 PM, Peter Eisentraut > wrote: > > How specifically would we do that? And what user would choose the > > behavior "start this background worker but don't worry if it doesn't work"? > > Well, if the background worker is auto-prewarm, you'd probably

Re: [HACKERS] [PATCH] Reduce src/test/recovery verbosity

2017-03-30 Thread Alvaro Herrera
Stephen Frost wrote: > I'd be fine with removing --verbose globally, as your patch does, but > there was some argument that we then would have long 'quiet' periods. > I haven't had a chance to go test if that's really the case yet though. Michael said that a running counter was displayed, which p

Re: [HACKERS] Schedule and Release Management Team for PG10

2017-03-30 Thread Alvaro Herrera
Robert Haas wrote: > - Alvaro proposes allowing more time next time, but not to change the > dates for this time. FWIW I didn't realize that the NY conference was ongoing, so count me for postponing the end of the current CF. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ Postgre

Re: [HACKERS] [bug fix] Savepoint-related statements terminates connection

2017-03-31 Thread Alvaro Herrera
Ashutosh Bapat wrote: > Please add this to the next commitfest. If this cannot be reproduced in 9.6, then it must be added to the Open Items wiki page instead. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services --

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-03-31 Thread Alvaro Herrera
Here's a version of this patch which I consider final. Thanks for your tips on using DSA. No crashes now. I am confused about not needing dsa_attach the second time around. If I do that, the dsa handle has been 0x7f'd, which I don't understand since it is supposed to be allocated in TopMemoryCo

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-04-01 Thread Alvaro Herrera
Robert Haas wrote: > On Tue, Mar 21, 2017 at 4:10 PM, Alvaro Herrera > wrote: > > Well, the number of work items is currently fixed; but if you have many > > BRIN indexes, then you'd overflow (lose requests). By using DSA I am > > making it easy to patch this af

Re: [HACKERS] brin autosummarization -- autovacuum "work items"

2017-04-01 Thread Alvaro Herrera
Alvaro Herrera wrote: > I also removed the behavior that on index creation the final partial > block range is always summarized. It's pointless. I just pushed this, without this change, because it breaks src/test/modules/brin. I still think it's pointless, but it'd requi

Re: [HACKERS] BRIN de-summarize ranges

2017-04-01 Thread Alvaro Herrera
ixed patch. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services >From d584e88386dd8c775a7b147b3dba328c26585858 Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Tue, 28 Feb 2017 01:45:21 -0300 Subject: [PATCH] Sup

Re: [HACKERS] BRIN de-summarize ranges

2017-04-01 Thread Alvaro Herrera
Alvaro Herrera wrote: > > However, I found that when calling brin_desummarize_range > > successively, an assertion is failed. It seems to me that it occurs > > when desummarizing a revmap page that is already desummarized. > > You're right, it's broken fo

Re: [HACKERS] Can't compile with profiling after BRIN autosummarization

2017-04-03 Thread Alvaro Herrera
Jeff Janes wrote: > This git bisects down to: > > commit 7526e10224f0792201e99631567bbe44492bbde4 > Author: Alvaro Herrera > Date: Sat Apr 1 14:00:53 2017 -0300 > > BRIN auto-summarization > > The lines are: > > extern void AutoVacuumReques

Re: [HACKERS] Can't compile with profiling after BRIN autosummarization

2017-04-03 Thread Alvaro Herrera
Jeff Janes wrote: > A newer gcc gave a better error message which lead me to add the line: > > #include "storage/block.h" /* for typedef BlockNumber */ > > into autovacuum.h, which fixes the problem. Ah, Peter already fixed it. -- Álvaro Herrerahttps://www.2ndQuadrant.

Re: [HACKERS] Refactoring identifier checks to consistently use strcmp

2017-04-03 Thread Alvaro Herrera
Daniel Gustafsson wrote: > Testing DefElem options is done with both strcmp() and pg_strcasecmp() a bit > mixed. Since the option defnames are all lowercased, either via IDENT, > keyword > rules or “by hand” with makeString(), using strcmp() is safe (and assumed to > be > so in quite a lot of pl

Re: [HACKERS] Re: extended stats not friendly towards ANALYZE with subset of columns

2017-04-05 Thread Alvaro Herrera
Noah Misch wrote: > The above-described topic is currently a PostgreSQL 10 open item. Álvaro, > since you committed the patch believed to have created it, you own this open > item. I'll commit a fix for this problem no later than Friday 14th. -- Álvaro Herrerahttps://www.2ndQua

Re: [HACKERS] Postgresql10 Bug report. (pg_catalog.pg_statistic_ext does not exist)

2017-04-06 Thread Alvaro Herrera
mark wrote: > m=# create table mytable (myid serial, mytext text); > CREATE TABLE > m=# \d mytable > ERROR: relation "pg_catalog.pg_statistic_ext" does not exist > LINE 8: FROM pg_catalog.pg_statistic_ext stat WHERE starelid = '163... > ^ Ah, what happens is you're using a new psql

Re: [HACKERS] BRIN cost estimate

2017-04-06 Thread Alvaro Herrera
Tom Lane wrote: > TBH, I think that code is in the noise. It doesn't involve any disk > access, or catalog access, or user-defined function calls. I wouldn't > bother trying to account for it. I removed it in the pushed version. > What you should be accounting for is the ensuing heap page acce

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Andres Freund wrote: > Unique Joins > - Tom's discussing things with David, not sure. This one was already included-and-removed from 9.6, Tom had said he'd give it priority during the current cycle as I recall. It seems unfair that it's still waiting for review on the last day of pg10's last com

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > On Fri, Apr 7, 2017 at 11:37 AM, Andres Freund wrote: > > Write Amplification Reduction Method (WARM) > > - fair number of people don't think it's ready for v10. Given the number of votes against putting this on pg10, I am going to back off from this patch now, with an ey

Re: [HACKERS] Remaining 2017-03 CF entries

2017-04-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > My offer to work with you on amcheck verification of WARM invariants > remains open. If nothing else, structuring things so that verification > is possible may clarify your design. Formalizing the preconditions, > postconditions, and legal states for on-disk structures mig

Re: [HACKERS] BRIN desummarization writes junk WAL records

2017-04-07 Thread Alvaro Herrera
Tom Lane wrote: > The proximate cause of the exception seems to be that > brinSetHeapBlockItemptr is being passed pagesPerRange = 0, > which is problematic since HEAPBLK_TO_REVMAP_INDEX tries to > divide by that. Looking one level down, the bogus value > seems to be coming out of an xl_brin_desum

Re: [HACKERS] Patch: Optimize memory allocation in function 'bringetbitmap'

2017-04-07 Thread Alvaro Herrera
Tomas Vondra wrote: > 1) bringetbitmap(PG_FUNCTION_ARGS) > > + /* > + * Estimate the approximate size of btup and allocate memory for btup. > + */ > + btupInitSize = bdesc->bd_tupdesc->natts * 16; > + btup = palloc(btupInitSize); > > What is the reasoning behind this estimate? I mean, this onl

Re: [HACKERS] mvstats triggers 32bit warnings

2017-04-07 Thread Alvaro Herrera
Andres Freund wrote: > Hi, > > compiling on linux 32 bit I get a lot of warnings due to printf format. > I suspect most of them should just be cured by using %zd or %zu instead > of %ld. You're right, they are. Confirmed, and pushed fix using %zd. I suppose %zu would be "more correct", but this

Re: [HACKERS] [COMMITTERS] pgsql: Improve 64bit atomics support.

2017-04-07 Thread Alvaro Herrera
Andres Freund wrote: > Improve 64bit atomics support. > > When adding atomics back in b64d92f1a, I added 64bit support as > optional; there wasn't yet a direct user in sight. That turned out to > be a bit short-sighted, it'd already have been useful a number of times. Seems like this killed an a

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-07 Thread Alvaro Herrera
Kyotaro HORIGUCHI wrote: > The attached patch is quiiiccck-and-dirty-hack of Michael's patch > just as a PoC of my proposal quoted above. This also passes the > 006 test. The major changes are the following. > > - Moved sync_above and truncted_to into RelationData. Interesting. I wonder if it

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-07 Thread Alvaro Herrera
I have claimed this patch as committer FWIW. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://ww

Re: [HACKERS] WAL logging problem in 9.4.3?

2017-04-07 Thread Alvaro Herrera
Alvaro Herrera wrote: > I suppose the rationale is that this shouldn't happen because any > operation that does things this way must hold an exclusive lock on the > relation. But that doesn't guarantee that the relcache entry is > completely stable, does it? If we can g

Re: [HACKERS] pg_stats_ext view does not seem all that useful

2017-04-10 Thread Alvaro Herrera
Tomas Vondra wrote: > On 04/10/2017 12:12 PM, David Rowley wrote: > > During my review and time spent working on the functional dependencies > > part of extended statistics I wondered what was the use for the > > pg_stats_ext view. I was unsure why the length of the serialised > > dependencies was

Re: [HACKERS] GCC 7 warnings

2017-04-11 Thread Alvaro Herrera
Tom Lane wrote: > Peter Eisentraut writes: > > d) Replace most of the problematic code with psprintf() and dynamically > > sized buffers. > > +1 for (c) as you have it. Later we might think about selectively > doing (d), but it seems like more work for probably not much benefit. Yeah -- also i

Re: [HACKERS] pg_stats_ext view does not seem all that useful

2017-04-13 Thread Alvaro Herrera
Tomas Vondra wrote: > On 04/10/2017 12:12 PM, David Rowley wrote: > > During my review and time spent working on the functional dependencies > > part of extended statistics I wondered what was the use for the > > pg_stats_ext view. I was unsure why the length of the serialised > > dependencies was

Re: [HACKERS] xmltable doc fix and example for XMLNAMESPACES

2017-04-13 Thread Alvaro Herrera
Pavel Stehule wrote: > Hi > > I am sending a patch with changes in XMLTABLE documentation proposed by > Arjen. Thanks, pushed with some rewording. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via p

Re: [HACKERS] Small doc fix for xmltable

2017-04-13 Thread Alvaro Herrera
Arjen Nienhuis wrote: > Hi, > > In the devel docs for xmltable there should be a comma after XMLNAMESPACES() Thanks, pushed. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing l

Re: [HACKERS] example for xmltable with XMLNAMESPACES

2017-04-13 Thread Alvaro Herrera
Arjen Nienhuis wrote: > It wasn't completely clear for me how to use namespaces in xmltable(). > Maybe add this to the documentation. It shows the default namespace > and quoting the namespace name. Thanks, pushed. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Developm

Re: [HACKERS] bugfix: xpath encoding issue

2017-04-13 Thread Alvaro Herrera
Pavel Stehule wrote: > Hi > > When I tested XMLTABLE function I found a bug of XPATH function - > xpath_internal > > There xmltype is not correctly encoded to xmlChar due possible invalid > encoding info in header. It is possible when XML was loaded with recv > function and has not UTF8 encoding.

Re: [HACKERS] pg_statistic_ext.staenabled might not be the best column name

2017-04-13 Thread Alvaro Herrera
Tom Lane wrote: > Peter Eisentraut writes: > > On 4/13/17 08:37, Heikki Linnakangas wrote: > >> We have a bunch of > 3-character prefixes already: amop*, amproc*, > >> enum*, cast*. But I think I nevertheless like "ste" better. > > > "stx" perhaps? > > > I would also be in favor of changing it

Re: [HACKERS] Allowing extended stats on foreign and partitioned tables

2017-04-14 Thread Alvaro Herrera
Noah Misch wrote: > On Mon, Apr 10, 2017 at 09:39:22PM +1200, David Rowley wrote: > > While reviewing extended stats I noticed that it was possible to > > create extended stats on many object types, including sequences. I > > mentioned that this should be disallowed. Statistics were then changed >

Re: [HACKERS] [pgsql-www] Small issue in online devel documentation build

2017-04-15 Thread Alvaro Herrera
Noah Misch wrote: > On Fri, Apr 14, 2017 at 09:38:32PM +0200, Magnus Hagander wrote: > > On Fri, Apr 14, 2017 at 9:36 PM, Peter Eisentraut > > wrote: > > > On 4/14/17 14:45, Magnus Hagander wrote: > > > > Attached is a patch that can be applied to pgweb which should fix > > > > all of > > >

Re: [HACKERS] Cutting initdb's runtime (Perl question embedded)

2017-04-15 Thread Alvaro Herrera
Andrew Dunstan wrote: > Alternatively, we could have an initdb TAP test that explicitly removed > the environment setting so we'd get coverage of select_default_timezone, > and have the buildfarm set TZ to something if it's not already set. What about having an initdb option that runs select_defa

Re: [HACKERS] Re: extended stats not friendly towards ANALYZE with subset of columns

2017-04-17 Thread Alvaro Herrera
Noah Misch wrote: > On Wed, Apr 05, 2017 at 08:58:54AM -0300, Alvaro Herrera wrote: > > Noah Misch wrote: > > > The above-described topic is currently a PostgreSQL 10 open item. Álvaro, > > > since you committed the patch believed to have created it, you own

Re: [HACKERS] extended stats not friendly towards ANALYZE with subset of columns

2017-04-17 Thread Alvaro Herrera
David Rowley wrote: > ERROR: extended statistics could not be collected for column "a" of > relation public.ab1 > HINT: Consider ALTER TABLE "public"."ab1" ALTER "a" SET STATISTICS -1 > > I don't think the error is useful here, as it means nothing gets done. > Probably better to just not (re)bu

Re: [HACKERS] On How To Shorten the Steep Learning Curve Towards PG Hacking...

2017-04-17 Thread Alvaro Herrera
Craig Ringer wrote: > Personally I have to agree that the learning curve is very steep. Some > of the docs and presentations help, but there's a LOT to understand. There is a wiki page "Developer_FAQ" which is supposed to help answer these questions. It is currently not very useful, because peop

Re: [HACKERS] Allowing extended stats on foreign and partitioned tables

2017-04-17 Thread Alvaro Herrera
David Rowley wrote: > While reviewing extended stats I noticed that it was possible to > create extended stats on many object types, including sequences. I > mentioned that this should be disallowed. Statistics were then changed > to be only allowed on plain tables and materialized views. > > This

Re: [HACKERS] pg_statistic_ext.staenabled might not be the best column name

2017-04-17 Thread Alvaro Herrera
Alvaro Herrera wrote: > Tom Lane wrote: > > Peter Eisentraut writes: > > > On 4/13/17 08:37, Heikki Linnakangas wrote: > > >> We have a bunch of > 3-character prefixes already: amop*, amproc*, > > >> enum*, cast*. But I think I nevertheles

Re: [HACKERS] extended stats not friendly towards ANALYZE with subset of columns

2017-04-17 Thread Alvaro Herrera
David Rowley wrote: > On 18 April 2017 at 05:12, Alvaro Herrera wrote: > > Pushed, after tweaking so that a WARNING is still emitted, because it's > > likely to be useful in pointing out a procedural mistake while extended > > stats are being tested. > > Th

Re: [HACKERS] Continuous buildfarm failures on hamster with bin-check

2017-04-18 Thread Alvaro Herrera
Tom Lane wrote: > Michael Paquier writes: > > That's the point I am trying to make upthread: slow buildfarm animals > > should have minimal impact on core code modifications. We could for > > example have one environment variable that lists all the parameters to > > modify in a single string and a

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Alvaro Herrera
Stas Kelvich wrote: > With patch MemoryContextStats() shows following hierarchy during slot > operations in > apply worker: > > TopMemoryContext: 83824 total in 5 blocks; 9224 free (8 chunks); 74600 used > ApplyContext: 8192 total in 1 blocks; 6520 free (4 chunks); 1672 used > ApplyMessag

Re: [HACKERS] Highly Variable Planning Times

2017-04-19 Thread Alvaro Herrera
Michael Malis wrote: > Hi, > > I've been encountering highly variable planning time on PG 9.5.4. > Running `EXPLAIN SELECT * FROM events_1171738` will take anywhere from > 200ms to 4s. This likely has to do with the table having 1300 partial > indexes on it (for reasons elaborated on in > https://

Re: [HACKERS] Unportable implementation of background worker start

2017-04-19 Thread Alvaro Herrera
Tom Lane wrote: > While I haven't yet tested it, it seems like a fix might be as simple > as deleting these lines in maybe_start_bgworker: > > /* > * Have ServerLoop call us again. Note that there might not > * actually *be* another runnable worker, but we d

Re: [HACKERS] Unportable implementation of background worker start

2017-04-19 Thread Alvaro Herrera
Tom Lane wrote: > Alvaro Herrera writes: > > Tom Lane wrote: > >> So I'm wondering what the design rationale was for only starting one > >> bgworker per invocation. > > > The rationale was that there may be other tasks waiting for postmaster > > att

Re: [HACKERS] [BUGS] Postgresql bug report - unexpected behavior of suppress_redundant_updates_trigger

2017-06-19 Thread Alvaro Herrera
Tom Lane wrote: > Robert Haas writes: > > On Mon, Jun 19, 2017 at 11:59 AM, Tom Lane wrote: > >> I don't think it's a bug, I think it's an intentional design tradeoff. > >> To suppress an update in this case, the trigger would have to grovel > >> through the individual fields and detoast them bef

Re: [HACKERS] GSoC 2017 weekly progress reports (week 3)

2017-06-19 Thread Alvaro Herrera
Shubham Barai wrote: > Project: Explicitly support predicate locks in index AMs besides b-tree > > Hi, > > > During this week, I continued my work on testing and created my first patch > for gist index. Please post it. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL D

Re: [BUGS] [HACKERS] Re: Postgresql bug report - unexpected behavior of suppress_redundant_updates_trigger

2017-06-19 Thread Alvaro Herrera
Tom Lane wrote: > As I mentioned upthread, it'd certainly be possible for the trigger > to iterate through the fields in a datatype-aware fashion and undo > compression or out-of-lineing before the comparison. But that would > eat a lot more cycles than the current implementation, and it seems >

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-06-19 Thread Alvaro Herrera
Mark Rofail wrote: > Okay, so major breakthrough. > > *Updates:* > >- The operator @>(anyarray, anyelement) is now functional > - The segmentation fault was due to applying PG_FREE_IF_COPY on a > datum when it should only be applied on TOASTed inputs > - The only problem now

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2017-06-20 Thread Alvaro Herrera
Satyanarayana Narlapuram wrote: > +1. > > This really helps PostgreSQL Azure service as well. When we are doing > the upgrades to the service, instead of abruptly terminating the > sessions we can provide this message. I think you mean "in addition to" rather than "instead of". Unless you have a

Re: [HACKERS] pg_terminate_backend can terminate background workers and autovacuum launchers

2017-06-21 Thread Alvaro Herrera
Yugo Nagata wrote: > However, we can terminate background workers by pg_terminate_backend. > In the following example, I terminated the logical replication launcher, > and this process did not appear again[1]. > > postgres=# select pg_terminate_backend(30902); > pg_terminate_backend >

Re: [HACKERS] Shortened URLs for commit messages

2017-06-21 Thread Alvaro Herrera
Bruce Momjian wrote: > Oh, here is a fixed version that requires an @ sign, which all message > id's have: > > sed '/http/!s;^\(Discussion: *\)\(.*@.*\)$;\1https://postgr.es/m/\2;' So how do you actually use this? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL D

Re: [HACKERS] [patch] pg_dump/pg_restore zerror() and strerror() mishap

2017-06-22 Thread Alvaro Herrera
Kunshchikov Vladimir wrote: Hi, > our testing team has noticed apparently wrong backup/restore error > messages like this: > > > pg_restore: [compress_io] could not read from input file: success > pg_dump: [directory archiver] could not write to output file: success > > > > Such "succes

Re: [HACKERS] GSoC 2017 Proposal for predicate locking in hash index

2017-06-22 Thread Alvaro Herrera
Shubham Barai wrote: > Hi, > > Now that hash index support write-ahead logging, it will be great if we add > support for predicate locking to it. > Implementation of predicate locking in hash index seems very simple. > I have already made changes in the code. I am currently working on testing. So

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-22 Thread Alvaro Herrera
Thomas Munro wrote: > Hmm. So the problem here is that AutoVacLauncherMain assumes that > there are only two possibilities: (1) there is no handle published in > shmem yet, so we should create a DSA area and publish the handle, and > (2) there is a handle published in shmem so we should attach to

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-22 Thread Alvaro Herrera
Yugo Nagata wrote: > Hi, > > As I report in another thread[1], I found the autovacuum launcher occurs > the following error in PG 10 when this received SIGINT. I can repuroduce > this by pg_cancel_backend or `kill -2 `. Thanks for the report, BTW! -- Álvaro Herrerahttps://www.2n

Re: [HACKERS] Fix comment in xlog.c

2017-06-22 Thread Alvaro Herrera
Amit Langote wrote: > Attached a patch for $SUBJECT. > > - * If RecPtr is not NULL, try to read a record at that position. Otherwise > + * If RecPtr is valid, try to read a record at that position. Otherwise > > Commit 4d6d425ab8d addressed the comment above XLogReadRecord() in > xlogreader.c,

Re: [HACKERS] Fix a typo in README.dependencies

2017-06-22 Thread Alvaro Herrera
Ashutosh Bapat wrote: > On Mon, Jun 5, 2017 at 8:22 AM, atorikoshi > wrote: > > Hi, > > > > I found below formula to compute selectivities, but > > I think the last Probability 'P(b=?)' should be 'P(c=?)'. > > > >> P(a=?,b=?,c=?) = P(a=?,b=?) * (d + (1-d)*P(b=?)) > > > > > > Attached patch fixes

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-22 Thread Alvaro Herrera
Thomas Munro wrote: > I thought about this when designing the DSA API. I couldn't think of > any good reason to provide an 'am-I-already-attached?' function > equivalent to dsm_find_mapping. It seemed to me that the client code > shouldn't ever be in any doubt about whether it's attached, and th

Re: [HACKERS] Setting pd_lower in GIN metapage

2017-06-23 Thread Alvaro Herrera
Masahiko Sawada wrote: > On Fri, Jun 23, 2017 at 3:31 PM, Michael Paquier > wrote: > > On Fri, Jun 23, 2017 at 3:17 PM, Amit Langote > > wrote: > >> Initially, I had naively set wal_consistency_check = all before running > >> make installcheck and then had to wait for a long time to confirm that

Re: [HACKERS] REPLICA IDENTITY FULL

2017-06-23 Thread Alvaro Herrera
Tom Lane wrote: > Peter Eisentraut writes: > > Any thoughts about keeping datumAsEqual() as a first check? I did some > > light performance tests, but it was inconclusive. > > Seems like it would tend to be a win if, in fact, the values are > usually equal. If they're usually not, then it's a l

Re: [HACKERS] REPLICA IDENTITY FULL

2017-06-23 Thread Alvaro Herrera
Andres Freund wrote: > On 2017-06-23 13:05:21 -0400, Alvaro Herrera wrote: > > Tom Lane wrote: > > > Peter Eisentraut writes: > > > > Any thoughts about keeping datumAsEqual() as a first check? I did some > > > > light performance tests, but it was inco

Re: [HACKERS] Logical replication: stuck spinlock at ReplicationSlotRelease

2017-06-23 Thread Alvaro Herrera
Albe Laurenz wrote: > Peter Eisentraut wrote: > > On 6/21/17 09:02, Albe Laurenz wrote: > >> 2017-06-21 14:55:12.033 CEST [23124] LOG: could not send data to client: > >> Broken pipe > >> 2017-06-21 14:55:12.033 CEST [23124] FATAL: connection to client lost > >> 2017-06-21 14:55:17.032 CEST [231

Re: [HACKERS] pg_terminate_backend can terminate background workers and autovacuum launchers

2017-06-23 Thread Alvaro Herrera
Yugo Nagata wrote: > I tried to make it. Patch attached. > > It is easy and simple. Although I haven't tried for autovacuum worker, > I confirmed I can change other process' parameters without affecting others. > Do you want this in PG? One advantage of this gadget is that you can signal backen

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-24 Thread Alvaro Herrera
Noah Misch wrote: > IMMEDIATE ATTENTION REQUIRED. This PostgreSQL 10 open item is long past due > for your status update. Please reacquaint yourself with the policy on open > item ownership[1] and then reply immediately. If I do not hear from you by > 2017-06-25 06:00 UTC, I will transfer this

Re: [HACKERS] Reducing pg_ctl's reaction time

2017-06-28 Thread Alvaro Herrera
Tom Lane wrote: > Andres Freund writes: > So when I removed the miscadmin.h include, I found out that pg_ctl is > also relying on PG_BACKEND_VERSIONSTR from that file. > > There are at least three things we could do here: > > 1. Give this up as not worth this much trouble. > > 2. Move PG_BACKE

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-28 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Jun 8, 2017 at 3:31 AM, Robert Haas wrote: > > I think if you're going to fix it so that we take spinlocks on > > MyWalSnd in a bunch of places that we didn't take them before, it > > would make sense to fix all the places where we're accessing those > > fields wi

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-29 Thread Alvaro Herrera
Noah Misch wrote: > On Wed, Jun 28, 2017 at 06:52:14PM -0400, Alvaro Herrera wrote: > > I think I'm done with the walsender half of this patch; I still need to > > review the walreceiver part. I will report back tomorrow 19:00 CLT. > > This PostgreSQL 10 open item i

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-06-30 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Jun 29, 2017 at 7:52 AM, Alvaro Herrera > wrote: > > I think I'm done with the walsender half of this patch; I still need to > > review the walreceiver part. I will report back tomorrow 19:00 CLT. > > Thanks! > > > Curre

Re: [HACKERS] Race-like failure in recovery/t/009_twophase.pl

2017-07-02 Thread Alvaro Herrera
Tom Lane wrote: > Part of the reason I'm confused is that the programming technique > being used in 009_twophase.pl, namely doing > > ($node_master, $node_slave) = ($node_slave, $node_master); > > and then working with the reversed variable names, is ENTIRELY TOO CUTE > FOR ITS OWN GOOD.

Re: [HACKERS] Race-like failure in recovery/t/009_twophase.pl

2017-07-02 Thread Alvaro Herrera
Tom Lane wrote: > I'd kind of like to fix it now, so I can reason in a less confused way > about the actual problem. OK, no objections here. > Last night I didn't have a clear idea of how > to make it better, but what I'm thinking this morning is: > > * Naming the underlying server objects "mas

Re: [HACKERS] WIP patch for avoiding duplicate initdb runs during "make check"

2017-07-03 Thread Alvaro Herrera
Tom Lane wrote: > (Other unfinished work: teaching the MSVC scripts to use this, > and teaching pg_upgrade's test script to use it.) Maybe it'd be simpler to rewrite pg_upgrade tests using PostgresNode instead? -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development

Re: [HACKERS] Race conditions with WAL sender PID lookups

2017-07-03 Thread Alvaro Herrera
Michael Paquier wrote: > Thanks Álvaro for pushing the patch. I had a second look and the > result looks good to me. > > - SpinLockAcquire(&walsnd->mutex); > + } > + pid = walsnd->pid; > The WAL receiver code used a cast to (int) in > pg_stat_get_wal_receiver(). I don't recall a

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-03 Thread Alvaro Herrera
Mark Rofail wrote: > On Mon, Jun 26, 2017 at 6:44 PM, Alexander Korotkov > wrote: > > > Have you met any particular problem here? Or is it just a lot of > > mechanical work? > > > > Just A LOT of mechanictal work, thankfully. The patch is now rebased and > all regress tests have passed (even th

Re: [HACKERS] Suspicious place in heap_prepare_freeze_tuple()

2017-07-05 Thread Alvaro Herrera
Teodor Sigaev wrote: > Playing around freezing tuple I found suspicious piece of code: > > heap_prepare_freeze_tuple(): > ... > frz->t_infomask = tuple->t_infomask; > ... > frz->t_infomask &= ~HEAP_XMAX_BITS; > frz->xmax = newxmax; > if (flags & FRM_MARK_COMMIT

Re: [HACKERS] More race conditions in logical replication

2017-07-05 Thread Alvaro Herrera
Noah Misch wrote: > The above-described topic is currently a PostgreSQL 10 open item. Peter, > since you committed the patch believed to have created it, you own this open > item. I volunteer to own this item. My next update is going to be on or before Friday 7th at 19:00 Chilean time, though I

Re: [HACKERS] Suspicious place in heap_prepare_freeze_tuple()

2017-07-05 Thread Alvaro Herrera
Masahiko Sawada wrote: > On Thu, Jul 6, 2017 at 1:36 AM, Alvaro Herrera > wrote: > > Teodor Sigaev wrote: > > > >> Playing around freezing tuple I found suspicious piece of code: > >> > >> heap_prepare_freeze_tuple(): >

Re: [HACKERS] [COMMITTERS] pgsql: pg_test_timing: Add NLS

2017-07-06 Thread Alvaro Herrera
Peter Eisentraut wrote: > pg_test_timing: Add NLS > > Also straighten out use of time unit abbreviations a bit. We (well, Carlos Chapi, who's doing the translation work now) just noticed that this has a bug in this line + printf("%6s %10s %10s\n", _("< us"), _("% of total"), _("count")); _(

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-07 Thread Alvaro Herrera
Peter Geoghegan wrote: > Here is the query: > > with recursive index_details as ( > select > 'pgbench_accounts_pkey'::text idx > ), [...] Hmm, this seems potentially very useful. Care to upload it to https://wiki.postgresql.org/wiki/Category:Snippets ? -- Álvaro Herrerah

Re: [HACKERS] More race conditions in logical replication

2017-07-07 Thread Alvaro Herrera
Petr Jelinek wrote: > So best idea I could come up with is to make use of the new condition > variable API. That lets us wait for variable which can be set per slot. > > It's not backportable however, I am not sure how big problem that is > considering the lack of complaints until now (maybe we c

Re: [HACKERS] RFC: Key normalization for nbtree

2017-07-10 Thread Alvaro Herrera
Claudio Freire wrote: > A missing optimization is that having tid unification allows VACUUM to > implement a different strategy when it needs to clean up only a tiny > fraction of the index. It can do the lookup by key-tid instead of > scanning the whole index, which can be a win if the index is l

Re: [HACKERS] More race conditions in logical replication

2017-07-10 Thread Alvaro Herrera
Alvaro Herrera wrote: > I'll next update this on or before Monday 10th at 19:00 CLT. I couldn't get to this today as I wanted. Next update on Wednesday 12th, same time. -- Álvaro Herrerahttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-10 Thread Alvaro Herrera
Amit Kapila wrote: > On Tue, Jul 11, 2017 at 6:51 AM, AP wrote: > > On Fri, Jul 07, 2017 at 05:58:25PM +0530, Amit Kapila wrote: > >> I can understand your concerns. To address first concern we need to > >> work on one or more of following work items: (a) work on vacuums that > >> can be trigger

Re: [HACKERS] New partitioning - some feedback

2017-07-10 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote > wrote: > > Actually, if \d had shown RELKIND_PARTITIONED_TABLE tables as of Type > > "partitioned table", we wouldn't need a separate flag for marking a table > > as having partitions. > > I think that is false. Whether someth

Re: [HACKERS] pgsql 10: hash indexes testing

2017-07-11 Thread Alvaro Herrera
Amit Kapila wrote: > Yes, I also think the same idea can be used, in fact, I have mentioned > it [1] as soon as you have committed that patch. Do we want to do > anything at this stage for PG-10? I don't think we should attempt > something this late unless people feel this is a show-stopper issu

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-11 Thread Alvaro Herrera
Mark Rofail wrote: >- now the RI checks utilise the @>(anyarray, anyelement) > - however there's a small problem: > operator does not exist: integer[] @> smallint > I assume that external casting would be required here. But how can I > downcast smallint to integer or in

Re: [HACKERS] New partitioning - some feedback

2017-07-12 Thread Alvaro Herrera
Amit Langote wrote: > On 2017/07/11 13:34, Alvaro Herrera wrote: > > Robert Haas wrote: > >> On Mon, Jul 10, 2017 at 2:15 AM, Amit Langote > >> wrote: > > > >>> Actually, if \d had shown RELKIND_PARTITIONED_TABLE tables as of Type > >>>

Re: [HACKERS] [WIP] Zipfian distribution in pgbench

2017-07-12 Thread Alvaro Herrera
Peter Geoghegan wrote: > Now, that might not seem like that much of a difference, but if you > consider how duplicates are handled in the B-Tree code, and how unique > index enforcement works, I think it could be. It could lead to heavy > buffer lock contention, because we sometimes do a lot of wo

Re: [HACKERS] More race conditions in logical replication

2017-07-12 Thread Alvaro Herrera
pment, 24x7 Support, Remote DBA, Training & Services >From 55533aa3cdc2fecbf7b6b6c661649342a204e73b Mon Sep 17 00:00:00 2001 From: Alvaro Herrera Date: Wed, 12 Jul 2017 18:38:33 -0400 Subject: [PATCH v3 1/1] Wait for slot to become free in when dropping it --- src/backend/replication/logic

Re: [HACKERS] Replacing lfirst() with lfirst_node() appropriately in planner.c

2017-07-13 Thread Alvaro Herrera
Ashutosh Bapat wrote: > Happened to stumble across some instances of lfirst() which could use > lfirst_node() in planner.c. Here's patch which replaces calls to > lfirst() extracting node pointers by lfirst_node() in planner.c. Sounds good. > Are we carrying out such replacements in master or th

Re: [HACKERS] PostgreSQL - Weak DH group

2017-07-13 Thread Alvaro Herrera
Michael Paquier wrote: > On Thu, Jul 13, 2017 at 5:32 PM, Heikki Linnakangas wrote: > > Objections to committing this now, instead of waiting for v11? > > But I am -1 for the sneak part. It is not the time to have a new > feature in 10, the focus is to stabilize. But if we were treating it as a

[HACKERS] bug in locking an update tuple chain

2017-07-14 Thread Alvaro Herrera
referenced table. It worked fine for them on 9.3.4. After some research, we determined that the problem disappeared if commit this commit was reverted: Author: Alvaro Herrera Branch: master Release: REL9_6_BR [533e9c6b0] 2016-07-15 14:17:20 -0400 Branch: REL9_5_STABLE Release: REL9_5_4 [649dd1b58] 2016

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-14 Thread Alvaro Herrera
Mark Rofail wrote: > On Wed, Jul 12, 2017 at 2:30 PM, Mark Rofail wrote: > > > On Wed, Jul 12, 2017 at 12:53 AM, Alvaro Herrera > > wrote: > >> > >> We have one opclass for each type combination -- int4 to int2, int4 to > >> int4, int4 to int8, etc.

Re: [HACKERS] More race conditions in logical replication

2017-07-15 Thread Alvaro Herrera
After going over this a bunch more times, I found other problems. For example, I noticed that if I create a temporary slot in one session, then another session would rightly go to sleep if it tries to drop that slot. But the slot goes away when the first session disconnects, so I'd expect the sle

Re: [HACKERS] More race conditions in logical replication

2017-07-18 Thread Alvaro Herrera
Alvaro Herrera wrote: > After going over this a bunch more times, I found other problems. For > example, I noticed that if I create a temporary slot in one session, > then another session would rightly go to sleep if it tries to drop that > slot. But the slot goes away when the f

Re: [HACKERS] GSoC 2017: Foreign Key Arrays

2017-07-18 Thread Alvaro Herrera
Mark Rofail wrote: > On Tue, 18 Jul 2017 at 7:43 pm, Alexander Korotkov > wrote: > > > separate operators for int4[] @>> int8, int4[] @>> int4, int4[] @>> int2, > > int4[] @>> numeric. > > > > My only comment on the separate operators is its high maintenance. Any new > datatype introduced a co

<    4   5   6   7   8   9   10   11   12   13   >