Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jeff Janes
On Monday, March 17, 2014, Tom Lane wrote: > Claudio Freire > writes: > > On Mon, Mar 17, 2014 at 7:01 PM, Jim Nasby > > wrote: > >> Even better would be if the planner could estimate how bad a plan will > >> become if we made assumptions that turn out to be wrong. > > > That's precisely what ris

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jeff Janes
On Monday, March 17, 2014, Atri Sharma wrote: > > > > On Mon, Mar 17, 2014 at 10:58 PM, Stephen Frost > > > wrote: > >> * Atri Sharma >> (atri.j...@gmail.com) >> wrote: >> > Isnt using a user given value for selectivity a pretty risky situation >> as >> > it can horribly screw up the plan sele

Re: [HACKERS] pg_archivecleanup bug

2014-03-17 Thread Amit Kapila
On Thu, Mar 13, 2014 at 11:18 AM, Bruce Momjian wrote: > > I have developed the attached patch which fixes all cases where > readdir() wasn't checking for errno, and cleaned up the syntax in other > cases to be consistent. 1. One common thing missed wherever handling for errno is added is below

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Tue, Mar 18, 2014 at 12:46 AM, Merlin Moncure wrote: > On Mon, Mar 17, 2014 at 1:45 PM, Pavel Stehule > wrote: > > I don't believe so SELECTIVITY can work well too. Slow queries are > usually > > related to some strange points in data. I am thinking so well concept > should > > be based on va

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Amit Kapila
On Tue, Mar 18, 2014 at 3:38 AM, Alvaro Herrera wrote: > Here's an adjusted version. In this one, the extra info is not used to > construct a string from pieces, but instead it puts it at the end, like > this: > > LOG: process 18899 still waiting for ShareLock on transaction 697 after > 1000.20

Re: [HACKERS] Triggers on foreign tables

2014-03-17 Thread Kouhei Kaigai
> I hacked on this for awhile, but there remain two matters on which I'm > uncertain about the right way forward. > > (1) To acquire the old tuple for UPDATE/DELETE operations, the patch closely > parallels our handling for INSTEAD OF triggers on views. It adds a wholerow > resjunk attribute, fro

Re: [HACKERS] [pgsql-advocacy] GSoC 2014 - mentors, students and admins

2014-03-17 Thread Fabrízio de Royes Mello
On Mon, Mar 17, 2014 at 10:43 PM, Stephen Frost wrote: > > * Stephen Frost (sfr...@snowman.net) wrote: > > I'm interested in mentoring and, unlike previous years, I've been > > collecting a personal list of things that I'd like to see worked on for > > PG which could be GSoC projects and will prov

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Amit Kapila
On Tue, Mar 18, 2014 at 5:51 AM, Tom Lane wrote: > Alvaro Herrera writes: >> 1. MyProcPort contains the database name; no need for the >> get_database_name() call in there. > > Wait. A. Minute. This patch wants to print the current database name in > the message? What on earth for? What other

Re: [HACKERS] Minimum supported version of Python?

2014-03-17 Thread David Johnston
Joshua D. Drake wrote > On 03/17/2014 07:31 PM, Peter Eisentraut wrote: >> >> On Sun, 2014-03-16 at 22:34 -0400, Tom Lane wrote: >>> Well, if you want to consider python 2.3 as supported, I have a >>> buildfarm >>> machine I am about to put online that has 2.3 on it. If I spin it up >>> with >>>

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Tom Lane
Robert Haas writes: > OK, I tried this out. The major complication that cropped up was > that, if we make the length word always a Size but align the buffer to > MAXIMUM_ALIGNOF, then the length word might get split if sizeof(Size) > > MAXIMUM_ALIGNOF. Hmm ... do we support any platforms where t

Re: [HACKERS] Minimum supported version of Python?

2014-03-17 Thread Tom Lane
On 03/17/2014 07:31 PM, Peter Eisentraut wrote: >> As I said, according to my testing, 2.3 is supported. If your >> experience is different, then please submit a reproducible bug report. It doesn't pass the regression tests. Do you need more of a bug report than that? >> There are many other fe

Re: [HACKERS] Minimum supported version of Python?

2014-03-17 Thread David Johnston
Peter Eisentraut-2 wrote > On Sat, 2014-03-15 at 20:55 -0400, Tom Lane wrote: >> Our documentation claims that the minimum Python version for plpython >> is 2.3. However, an attempt to build with that on an old Mac yielded >> a bunch of failures in the plpython_types regression test, > > It has f

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 1:26 PM, Robert Haas wrote: > Oh, yeah. Duh. Clearly my brain isn't working today. Hmm, so maybe > this will be fairly simple... will try it out. OK, I tried this out. The major complication that cropped up was that, if we make the length word always a Size but align t

Re: [HACKERS] Minimum supported version of Python?

2014-03-17 Thread Joshua D. Drake
On 03/17/2014 07:31 PM, Peter Eisentraut wrote: > > On Sun, 2014-03-16 at 22:34 -0400, Tom Lane wrote: >> Well, if you want to consider python 2.3 as supported, I have a buildfarm >> machine I am about to put online that has 2.3 on it. If I spin it up with >> python enabled, I expect you to see

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-17 Thread Kouhei Kaigai
> > Also, I split the patches again because *demonstration* part is much > > larger than the patches to the core backend. It will help reviewing. > > * pgsql-v9.4-vacuum_page_hook.v11.patch > > -> It adds a hook for each page being vacuumed; that needs to synchronize > > the status of in-memor

Re: [HACKERS] Minimum supported version of Python?

2014-03-17 Thread Peter Eisentraut
On Sun, 2014-03-16 at 22:34 -0400, Tom Lane wrote: > Well, if you want to consider python 2.3 as supported, I have a buildfarm > machine I am about to put online that has 2.3 on it. If I spin it up with > python enabled, I expect you to see to it that it starts passing. If you > won't do that, I'

Re: [HACKERS] Is this a bug?

2014-03-17 Thread Michael Paquier
On Tue, Mar 18, 2014 at 10:24 AM, Fabrízio de Royes Mello wrote: > > On Thu, Mar 13, 2014 at 10:22 AM, Robert Haas wrote: >> Well, it's fairly harmless, but it might not be a bad idea to tighten that >> up. > The attached patch tighten that up. Hm... It might be interesting to include it in 9.4 I

Re: contrib/cache_scan (Re: [HACKERS] What's needed for cache-only table scan?)

2014-03-17 Thread Haribabu Kommi
On Mon, Mar 17, 2014 at 11:45 AM, Kouhei Kaigai wrote: > Hello, > > The attached patches are revised ones according to the latest custom-plan > interface patch (v11). > The cache-scan module was re-implemented on the newer interface, and also > I noticed the extension does not handle the tuples be

Re: [HACKERS] Wiki Page Draft for upcoming release

2014-03-17 Thread David Johnston
Josh Berkus wrote > All, > > https://wiki.postgresql.org/wiki/20140320UpdateIssues > > I'm sure my explanation of the data corruption issue is not correct, so > please fix it. Thanks! I presume that because there is no way the master could have sent bad table data to the replication slaves that

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Alvaro Herrera
Tom Lane escribió: > Alvaro Herrera writes: > > 1. MyProcPort contains the database name; no need for the > > get_database_name() call in there. > > Wait. A. Minute. This patch wants to print the current database name in > the message? What on earth for? What other error messages do you see >

Re: [HACKERS] Wiki Page Draft for upcoming release

2014-03-17 Thread David Johnston
I sent a post to -general with a much more detailed brain dump of my current understanding on this topic. The main point I'm addressing here is how to recover from this problem. Since a symptom of the problem is that pg_dump/restore can fail saying that (in some instances) the only viable restore

Re: [HACKERS] [pgsql-advocacy] GSoC 2014 - mentors, students and admins

2014-03-17 Thread Stephen Frost
* Stephen Frost (sfr...@snowman.net) wrote: > I'm interested in mentoring and, unlike previous years, I've been > collecting a personal list of things that I'd like to see worked on for > PG which could be GSoC projects and will provide such in the next few > days to this list (unless there's a dif

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/17/2014 05:55 PM, Jeff Janes wrote: > On Mon, Mar 17, 2014 at 5:48 PM, Craig Ringer > > LOCK TABLE table1, table2, table3, ... > > would help, instead of doing individual statements? > > If I recall correctly, someone did submit a patch to do

Re: [HACKERS] Is this a bug?

2014-03-17 Thread Fabrízio de Royes Mello
On Thu, Mar 13, 2014 at 10:22 AM, Robert Haas wrote: > > On Wed, Mar 12, 2014 at 11:11 PM, Fabrízio de Royes Mello > wrote: > > Hi all, > > > > Shouldn't the "ALTER" statements below raise an exception? > > > > fabrizio=# CREATE TABLE foo(bar SERIAL PRIMARY KEY); > > CREATE TABLE > > > > fabrizio

[HACKERS] commit fest 2014-01 closed

2014-03-17 Thread Peter Eisentraut
I have closed the current commit fest. This doesn't have to mean much yet, except that it's probably time to stop reviewing and discussing, and instead focus on committing and polishing what is already committed. In this commit fest, we have committed 53 out of 112 patches, so that wasn't so bad.

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Jeff Janes
On Mon, Mar 17, 2014 at 5:48 PM, Craig Ringer wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 03/18/2014 07:20 AM, Joe Conway wrote: > > On 03/17/2014 04:15 PM, Tom Lane wrote: > >> Jim Nasby writes: > >>> On 3/17/14, 8:47 AM, Tom Lane wrote: > (Note that this is only one of

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Craig Ringer
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/18/2014 07:20 AM, Joe Conway wrote: > On 03/17/2014 04:15 PM, Tom Lane wrote: >> Jim Nasby writes: >>> On 3/17/14, 8:47 AM, Tom Lane wrote: (Note that this is only one of assorted O(N^2) behaviors in older versions of pg_dump; we've g

[HACKERS] Wiki Page Draft for upcoming release

2014-03-17 Thread Josh Berkus
All, https://wiki.postgresql.org/wiki/20140320UpdateIssues I'm sure my explanation of the data corruption issue is not correct, so please fix it. Thanks! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Andres Freund
On 2014-03-17 20:21:18 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > 1. MyProcPort contains the database name; no need for the > > get_database_name() call in there. > > Wait. A. Minute. This patch wants to print the current database name in > the message? What on earth for? What other e

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Tom Lane
Alvaro Herrera writes: > 1. MyProcPort contains the database name; no need for the > get_database_name() call in there. Wait. A. Minute. This patch wants to print the current database name in the message? What on earth for? What other error messages do you see doing that? It should print a ta

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Alvaro Herrera
Alvaro Herrera escribió: > Another thing that jumped at me is that passing a TID but not a Relation > is fairly useless as it stands. We might try to add some more stuff > later, such as printing tuple contents as previous versions of the patch > did, but given the opposition the idea had previou

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 19:55:01 -0400, Tom Lane wrote: >>> I think the best way to really cleanup a table is to use something like: >>> ALTER TABLE rew ALTER COLUMN data TYPE text USING (data); >> Um... don't we have logic in there that's smart enough to short-circuit >> that? > N

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 19:58:18 -0400, Tom Lane wrote: > Alvaro Herrera writes: > > Andres Freund wrote: > >> I think the best way to really cleanup a table is to use something like: > >> ALTER TABLE rew ALTER COLUMN data TYPE text USING (data); > >> where text is the previous type of the column. That shou

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Alvaro Herrera writes: > Andres Freund wrote: >> I think the best way to really cleanup a table is to use something like: >> ALTER TABLE rew ALTER COLUMN data TYPE text USING (data); >> where text is the previous type of the column. That should trigger a >> full table rewrite, without any finesse

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 19:55:01 -0400, Tom Lane wrote: > > I think the best way to really cleanup a table is to use something like: > > ALTER TABLE rew ALTER COLUMN data TYPE text USING (data); > > where text is the previous type of the column. That should trigger a > > full table rewrite, without any fines

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 20:51:31 -0300, Alvaro Herrera wrote: > > I think the best way to really cleanup a table is to use something like: > > ALTER TABLE rew ALTER COLUMN data TYPE text USING (data); > > where text is the previous type of the column. That should trigger a > > full table rewrite, without any

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 21:09:10 +, Greg Stark wrote: >> That said, it would be nice to actually fix the problem, not just >> detect it. Eventually vacuum would fix the problem. I think. I'm not >> really sure what will happen actually. > Indexes will quite possibly stay corrupt

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
Claudio Freire writes: > On Mon, Mar 17, 2014 at 7:01 PM, Jim Nasby wrote: >> Even better would be if the planner could estimate how bad a plan will >> become if we made assumptions that turn out to be wrong. > That's precisely what risk estimation was about. Yeah. I would like to see the plan

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-03-17 21:09:10 +, Greg Stark wrote: > > That said, it would be nice to actually fix the problem, not just > > detect it. Eventually vacuum would fix the problem. I think. I'm not > > really sure what will happen actually. > > Indexes will quite possibly stay corr

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-17 Thread Alvaro Herrera
Here's an adjusted version. In this one, the extra info is not used to construct a string from pieces, but instead it puts it at the end, like this: LOG: process 18899 still waiting for ShareLock on transaction 697 after 1000.203 ms CONTEXT: while operating on tuple (0,2) in relation "public".

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 21:09:10 +, Greg Stark wrote: > That said, it would be nice to actually fix the problem, not just > detect it. Eventually vacuum would fix the problem. I think. I'm not > really sure what will happen actually. Indexes will quite possibly stay corrupted. I think. If there was a in

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/17/2014 04:15 PM, Tom Lane wrote: > Jim Nasby writes: >> On 3/17/14, 8:47 AM, Tom Lane wrote: >>> (Note that this is only one of assorted O(N^2) behaviors in >>> older versions of pg_dump; we've gradually stamped them out >>> over time.) > >> O

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Tom Lane
Jim Nasby writes: > On 3/17/14, 8:47 AM, Tom Lane wrote: >> (Note that this is only one of assorted O(N^2) behaviors in older versions >> of pg_dump; we've gradually stamped them out over time.) > On that note, it's recommended that when you are taking a backup to restore > into a newer version

Re: [HACKERS] pg_dump without explicit table locking

2014-03-17 Thread Jim Nasby
On 3/17/14, 8:47 AM, Tom Lane wrote: Pavel Stehule writes: 2014-03-17 12:52 GMT+01:00 Jürgen Strobel : I've googled the problem and there seem to be more people with similar problems, so I made this a command line option --no-table-locks and wrapped it up in as nice a patch against github/mas

Re: [HACKERS] AUTOCOMMIT off + ON_ERROR_ROLLBACK usability

2014-03-17 Thread Noah Misch
On Sun, Mar 16, 2014 at 11:58:53AM +, Greg Stark wrote: > I've been putting my money where my mouth is and running with > AUTOCOMMIT=off and ON_ERROR_ROLLBACK set which I've been recommending > for years but hadn't gotten around to switching to myself. > > I think we knew the user experience w

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 5:15 PM, Josh Berkus wrote: > So, if we're going to support query decorators, we might as well go all > the way and just offer Oracle-style "use this index". Speaking as > someone who is often called on to fix performance issues in other > people's databases, I find major

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Claudio Freire
On Mon, Mar 17, 2014 at 7:12 PM, Jim Nasby wrote: > On 3/17/14, 5:07 PM, Claudio Freire wrote: > > >> On Mon, Mar 17, 2014 at 7:01 PM, Jim Nasby > j...@nasby.net>> wrote: >> >> Even better would be if the planner could estimate how bad a plan >> will become if we made assumptions that turn ou

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Vik Fearing
On 03/17/2014 11:15 PM, Josh Berkus wrote: > 2. they eventually become a barrier to upgrading, once the performance > of the DB engine changes in a way that makes older query decorators > crippling and/or erroneous. Because they are scattered all around the > code, it then becomes a major refactor

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Vik Fearing
On 03/17/2014 06:28 PM, Stephen Frost wrote: > a value passed in > as part of the query, with a high likelihood of users figuring out how > to use it to say "use my plan forever and always".. Last time this came up, I said on irc that if we ever do implement hints, I'd like them to be tied to a ma

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Josh Berkus
On 03/17/2014 01:57 PM, Martijn van Oosterhout wrote: > On Mon, Mar 17, 2014 at 01:20:47PM -0500, Merlin Moncure wrote: >> A query plan is a complicated thing that is the result of detail >> analysis of the data. I bet there are less than 100 users on the >> planet with the architectural knowledge

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jim Nasby
On 3/17/14, 5:12 PM, Claudio Freire wrote: On Mon, Mar 17, 2014 at 7:09 PM, Jim Nasby mailto:j...@nasby.net>> wrote: On 3/17/14, 2:16 PM, Merlin Moncure wrote: On Mon, Mar 17, 2014 at 1:45 PM, Pavel Stehulemailto:pavel.steh...@gmail.com>> wrote: >I don't believe so S

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jim Nasby
On 3/17/14, 5:07 PM, Claudio Freire wrote: On Mon, Mar 17, 2014 at 7:01 PM, Jim Nasby mailto:j...@nasby.net>> wrote: Even better would be if the planner could estimate how bad a plan will become if we made assumptions that turn out to be wrong. That's precisely what risk estimation was a

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Claudio Freire
On Mon, Mar 17, 2014 at 7:09 PM, Jim Nasby wrote: > On 3/17/14, 2:16 PM, Merlin Moncure wrote: > >> On Mon, Mar 17, 2014 at 1:45 PM, Pavel Stehule >> wrote: >> >>> >I don't believe so SELECTIVITY can work well too. Slow queries are >>> usually >>> >related to some strange points in data. I am th

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jim Nasby
On 3/17/14, 2:16 PM, Merlin Moncure wrote: On Mon, Mar 17, 2014 at 1:45 PM, Pavel Stehule wrote: >I don't believe so SELECTIVITY can work well too. Slow queries are usually >related to some strange points in data. I am thinking so well concept should >be based on validity of estimations. Some p

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Claudio Freire
On Mon, Mar 17, 2014 at 7:01 PM, Jim Nasby wrote: > Even better would be if the planner could estimate how bad a plan will > become if we made assumptions that turn out to be wrong. > That's precisely what risk estimation was about. Something like SELECT * FROM wherever WHEN id > something LIM

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jim Nasby
On 3/17/14, 3:32 PM, Merlin Moncure wrote: On Mon, Mar 17, 2014 at 2:02 PM, Jim Nasby wrote: Just being able to detect that something has possibly gone wrong would be useful. We could log that to alert the DBA/user of a potential bad plan. We could even format this in such a fashion that it's s

Re: [HACKERS] warning when compiling utils/tqual.h

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 12:55 PM, Andres Freund wrote: > On 2014-03-17 12:50:37 -0400, Tom Lane wrote: >> Alvaro Herrera writes: >> > I noticed (by running "cd src/include ; make check" with the attached >> > patch applied) that since commit b89e151054 ("Introduce logical >> > decoding.") tqual.h

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Greg Stark
On Mon, Mar 17, 2014 at 3:28 PM, Tom Lane wrote: > I'm thinking we'd better promote that Assert to a normal runtime elog. I wasn't sure about this but on further thought I think it's a really good idea and should be mentioned in the release notes. One of the things that's been bothering me about

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Martijn van Oosterhout
On Mon, Mar 17, 2014 at 01:20:47PM -0500, Merlin Moncure wrote: > A query plan is a complicated thing that is the result of detail > analysis of the data. I bet there are less than 100 users on the > planet with the architectural knowledge of the planner to submit a > 'plan'. What users do have i

Re: [HACKERS] bpchar functinos

2014-03-17 Thread Noah Misch
On Sat, Mar 15, 2014 at 05:02:44PM +0330, Mohsen SM wrote: > I want to fined when is used these functions(what query caused the call of > these functions) : > -char_bpchar() > -bpchar_name() > -name_bpchar() They implement casts. For example, "select 'foo'::character(10)::name" calls bpchar_name(

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 16:17:35 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > On 2014-03-17 14:01:03 -0400, Tom Lane wrote: > > > Andres Freund writes: > > > > * I wonder if we should make the possible origins a bit more > > > > general as it's perfectly possible to trigger the problem without

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 2:02 PM, Jim Nasby wrote: > Just being able to detect that something has possibly gone wrong would be > useful. We could log that to alert the DBA/user of a potential bad plan. We > could even format this in such a fashion that it's suitable for emailing the > community wit

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Jim Nasby
On 3/17/14, 12:58 PM, Stephen Frost wrote: * Merlin Moncure (mmonc...@gmail.com) wrote: Yeah -- the most common case I see is outlier culling where several repeated low non-deterministic selectivity quals stack reducing the row count estimate to 1. For example: SELECT * FROM foo WHERE length(ba

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Alvaro Herrera writes: > Uhm. But at the bottom of that block, right above the "failed:" label > (heapam.c line 4527 in current master), we recheck the tuple for > "locked-only-ness"; and fail the whole operation by returning > HeapTupleUpdated, if it's not locked-only, no? Which would cause > E

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Alvaro Herrera
Andres Freund wrote: > On 2014-03-17 14:01:03 -0400, Tom Lane wrote: > > Andres Freund writes: > > > * I wonder if we should make the possible origins a bit more > > > general as it's perfectly possible to trigger the problem without > > > foreign keys. Maybe: "can arise when a table row that

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 1:45 PM, Pavel Stehule wrote: > I don't believe so SELECTIVITY can work well too. Slow queries are usually > related to some strange points in data. I am thinking so well concept should > be based on validity of estimations. Some plans are based on totally wrong > estimatio

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
Atri Sharma writes: > Wont this have scaling issues and issues over time as the data in the > table changes? It can't possibly have worse problems of that sort than explicitly specifying a plan does. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 14:29:56 -0400, Tom Lane wrote: >> [ scratches head ... ] If that's what's happening, isn't it a bug in >> itself? Surely the WAL record ought to point at the tuple that was >> locked. > There's a separate XLOG_HEAP2_LOCK_UPDATED record, for every later tup

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:52:25 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-03-17 14:29:56 -0400, Tom Lane wrote: > >> [ scratches head ... ] If that's what's happening, isn't it a bug in > >> itself? Surely the WAL record ought to point at the tuple that was > >> locked. > > > There's a

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Pavel Stehule
2014-03-17 19:35 GMT+01:00 Atri Sharma : > > > > On Mon, Mar 17, 2014 at 11:50 PM, Merlin Moncure wrote: > >> On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma >> wrote: >> > >> >> There's a big difference between saying to the planner, "Use plan X" >> >> vs "Here's some information describing the da

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 11:50 PM, Merlin Moncure wrote: > On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma wrote: > > > >> There's a big difference between saying to the planner, "Use plan X" > >> vs "Here's some information describing the data supporting choosing > >> plan X intelligently". The l

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:29:56 -0400, Tom Lane wrote: > Andres Freund writes: > > To me that looks sufficient to trigger the bug, because we're issuing a > > wal record about the row that was passed to heap_lock_update(), not the > > latest one in the ctid chain. When replaying that record, it will reset

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > To me that looks sufficient to trigger the bug, because we're issuing a > wal record about the row that was passed to heap_lock_update(), not the > latest one in the ctid chain. When replaying that record, it will reset > the t_ctid field, thus breaking the chain. [ scratc

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:16:41 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-03-17 14:01:03 -0400, Tom Lane wrote: > >> IIUC, this case only occurs when using the new-in-9.3 types of > >> nonexclusive row locks. I'm willing to bet that the number of > >> applications using those is negligibl

Re: [HACKERS] jsonb status

2014-03-17 Thread Oleg Bartunov
Alexander will take a look on TriConsistent function. On Mon, Mar 17, 2014 at 9:48 PM, Andrew Dunstan wrote: > > On 03/16/2014 04:10 AM, Peter Geoghegan wrote: >> >> On Thu, Mar 13, 2014 at 2:00 PM, Andrew Dunstan >> wrote: >>> >>> I'll be travelling a good bit of tomorrow (Friday), but I hope P

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 12:57 PM, Atri Sharma wrote: > >> There's a big difference between saying to the planner, "Use plan X" >> vs "Here's some information describing the data supporting choosing >> plan X intelligently". The latter allows for better plans in the face >> of varied/changing data

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 14:01:03 -0400, Tom Lane wrote: >> IIUC, this case only occurs when using the new-in-9.3 types of >> nonexclusive row locks. I'm willing to bet that the number of >> applications using those is negligible; so I think it's all right to not >> mention that case

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 14:01:03 -0400, Tom Lane wrote: > Andres Freund writes: > > * I wonder if we should make the possible origins a bit more > > general as it's perfectly possible to trigger the problem without > > foreign keys. Maybe: "can arise when a table row that has been updated > > is row l

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > That's much better, yes. Two things: > * I'd change the warning about unique key violations into a more general > one about constraints. Foreign key and exclusion constraint are also > affected... I'll see what I can do. > * I wonder if we should make the possible or

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
There's a big difference between saying to the planner, "Use plan X" > vs "Here's some information describing the data supporting choosing > plan X intelligently". The latter allows for better plans in the face > of varied/changing data, integrates with the planner in natural way, > and encourages

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Merlin Moncure (mmonc...@gmail.com) wrote: > Yeah -- the most common case I see is outlier culling where several > repeated low non-deterministic selectivity quals stack reducing the > row count estimate to 1. For example: > SELECT * FROM foo WHERE length(bar) <= 1000 AND length(bar) >= 2; This

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 11:28:45 -0400, Tom Lane wrote: > Hm ... "rows disappearing from indexes" might make people think that > they could fix or mitigate the damage via REINDEX. Good point. I guess in some cases it will end up working because VACUUM/hot pruning have cleaned up the mess, but that's certain

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 13:42:59 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: > >> First, see suggested text in my first-draft release announcement. > > > I don't think that text is any better, it's imo even wrong: > > "The bug causes rows to vanish fr

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 10:54 PM, Tom Lane wrote: > Atri Sharma writes: > > On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane wrote: > >> Possibly worth noting is that in past discussions, we've concluded that > >> the most sensible type of hint would not be "use this plan" at all, but > >> "here's wha

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Merlin Moncure
On Mon, Mar 17, 2014 at 11:15 AM, Tom Lane wrote: > David Johnston writes: >> Need to discuss the general "why" before any meaningful help on the "how" is >> going to be considered by hackers. > > Possibly worth noting is that in past discussions, we've concluded that > the most sensible type of

Re: [HACKERS] jsonb status

2014-03-17 Thread Andrew Dunstan
On 03/16/2014 04:10 AM, Peter Geoghegan wrote: On Thu, Mar 13, 2014 at 2:00 PM, Andrew Dunstan wrote: I'll be travelling a good bit of tomorrow (Friday), but I hope Peter has finished by the time I am back on deck late tomorrow and that I am able to commit this on Saturday. I asked Andrew to

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
> There's lots of ways to implement planner hints, but I fail to see the > point in discussing how to implement something we actively don't want. > > > +1. The original poster wanted a way to implement it as a personal project or something ( I think he only replied to me, not the entire list). Pla

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
The larger question to answer first is whether we want to implement > something that is deterministic... > > How about just dropping the whole concept of "hinting" and provide a way > for > someone to say "use this plan, or die trying." Maybe require it be used in > conjunction with named PREPARE

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Tom Lane
Andres Freund writes: > On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: >> First, see suggested text in my first-draft release announcement. > I don't think that text is any better, it's imo even wrong: > "The bug causes rows to vanish from indexes during recovery due to > simultaneous updates o

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: > Of course, this is not a nice hack. Specifically after our discussion on > IRC the other day, I am against planner hints, but if we are just > discussing how it could be done, I could think of some ways which I listed. There's lots of ways to implement

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Tom Lane
Robert Haas writes: > After mulling over a few possible approaches, I came up with the > attached, which seems short and to the point. Looks reasonable in principle. I didn't run through all the existing PGSharedMemoryDetach calls to see if there are any other places to call dsm_detach_all, but

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 10:58 PM, Stephen Frost wrote: > * Atri Sharma (atri.j...@gmail.com) wrote: > > Isnt using a user given value for selectivity a pretty risky situation as > > it can horribly screw up the plan selection? > > > > Why not allow the user to specify an alternate plan and have t

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-15 16:02:19 -0400, Tom Lane wrote: > First-draft release notes are committed, and should be visible at > http://www.postgresql.org/docs/devel/static/release-9-3-4.html > once guaibasaurus does its next buildfarm run a few minutes from > now. Any suggestions? So, the current text is: "T

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread David Johnston
Atri Sharma wrote > On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane < > tgl@.pa > > wrote: > >> David Johnston < > polobo@ > > writes: >> > Need to discuss the general "why" before any meaningful help on the >> "how" is >> > going to be considered by hackers. >> >> Possibly worth noting is that in p

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Stephen Frost
* Atri Sharma (atri.j...@gmail.com) wrote: > Isnt using a user given value for selectivity a pretty risky situation as > it can horribly screw up the plan selection? > > Why not allow the user to specify an alternate plan and have the planner Uh, you're worried about the user given us a garbage s

Re: [HACKERS] Portability issues in shm_mq

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 12:03 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, Mar 16, 2014 at 10:45 PM, Tom Lane wrote: >>> Well, it will result in padding space when you maxalign the length word, >>> but I don't see why it wouldn't work; and it would certainly be no less >>> efficient than

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Tom Lane
Atri Sharma writes: > On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane wrote: >> Possibly worth noting is that in past discussions, we've concluded that >> the most sensible type of hint would not be "use this plan" at all, but >> "here's what to assume about the selectivity of this WHERE clause". >> Th

Re: [HACKERS] on_exit_reset fails to clear DSM-related exit actions

2014-03-17 Thread Robert Haas
On Mon, Mar 17, 2014 at 11:32 AM, Tom Lane wrote: > Robert Haas writes: >> One option is to just change that function to also unmap the control >> segment, and maybe rename it to dsm_detach_all(), and then use that >> everywhere. The problem is that I'm not sure we really want to incur >> the ov

Re: [HACKERS] Planner hints in Postgresql

2014-03-17 Thread Atri Sharma
On Mon, Mar 17, 2014 at 9:45 PM, Tom Lane wrote: > David Johnston writes: > > Need to discuss the general "why" before any meaningful help on the > "how" is > > going to be considered by hackers. > > Possibly worth noting is that in past discussions, we've concluded that > the most sensible type

Re: [HACKERS] First-draft release notes for next week's releases

2014-03-17 Thread Andres Freund
On 2014-03-17 10:03:52 -0700, Josh Berkus wrote: > On 03/17/2014 08:28 AM, Tom Lane wrote: > > Greg Stark writes: > >> The error causes some rows to disappear from indexes resulting in > >> inconsistent query results on a hot standby depending on whether > >> indexes are used. If the standby is su

  1   2   >