Re: [HACKERS] [PATCH] Use MAP_HUGETLB where supported (v3)

2013-10-29 Thread Sergey Konoplev
On Tue, Oct 29, 2013 at 9:31 PM, Tom Lane wrote: > Sergey Konoplev writes: >> On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen >> wrote: >>> This is a slightly reworked version of the patch submitted by Richard >>> Poole last month, which was based on Christian Kruse's earlier patch. > >> Is

Re: [HACKERS] Something fishy happening on frogmouth

2013-10-29 Thread Amit Kapila
On Wed, Oct 30, 2013 at 12:42 AM, Tom Lane wrote: > The last two buildfarm runs on frogmouth have failed in initdb, > like this: > > creating directory > d:/mingw-bf/root/HEAD/pgsql.2492/src/test/regress/./tmp_check/data ... ok > creating subdirectories ... ok > selecting default max_connections

Re: [HACKERS] [PATCH] Use MAP_HUGETLB where supported (v3)

2013-10-29 Thread Abhijit Menon-Sen
At 2013-10-24 19:00:28 +0200, and...@2ndquadrant.com wrote: > > I think we should log when we tried to use hugepages but fell back to > plain mmap, currently it's hard to see whether they are used. Good idea, thanks. I'll do this in the next patch I post (which will be after we reach some consensu

Re: [HACKERS] [PATCH] Use MAP_HUGETLB where supported (v3)

2013-10-29 Thread Abhijit Menon-Sen
At 2013-10-24 16:06:19 +0300, hlinnakan...@vmware.com wrote: > > Let's get rid of the rounding. I share Andres's concern that the bug is present in various recent kernels that are going to stick around for quite some time. Given the rather significant performance gain, I think it's worth doing som

Re: [HACKERS] [PATCH] Use MAP_HUGETLB where supported (v3)

2013-10-29 Thread Tom Lane
Sergey Konoplev writes: > On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen > wrote: >> This is a slightly reworked version of the patch submitted by Richard >> Poole last month, which was based on Christian Kruse's earlier patch. > Is it possible that this patch will be included in a minor v

Re: [HACKERS] [PATCH] Use MAP_HUGETLB where supported (v3)

2013-10-29 Thread Sergey Konoplev
Hi, On Wed, Oct 23, 2013 at 11:03 PM, Abhijit Menon-Sen wrote: > This is a slightly reworked version of the patch submitted by Richard > Poole last month, which was based on Christian Kruse's earlier patch. Is it possible that this patch will be included in a minor version of 9.3? IMHO hugepage

Re: [HACKERS] Re: Using indexes for ORDER BY and PARTITION BY clause in windowing functions

2013-10-29 Thread Kyotaro HORIGUCHI
Hello, > > With this index, you will get a different plan like this, > > > Exactly my point, can we look at making windowing functions > smart and make use of available indexes? I might have guessed.. > > Does this satisfies your needs? > > > Not exactly. If I have missed to mention, this is no

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Craig Ringer
On 10/30/2013 11:25 AM, Kohei KaiGai wrote: > 2013/10/30 Craig Ringer : >> On 10/30/2013 10:50 AM, Tom Lane wrote: >>> Craig Ringer writes: > I'd kind of like to see FK constraints affected by RLS for > non-superusers, at least as an option. >>> I think that's a complete nonstarter. Aside

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Kohei KaiGai
2013/10/30 Craig Ringer : > On 10/30/2013 10:50 AM, Tom Lane wrote: >> Craig Ringer writes: >>> > I'd kind of like to see FK constraints affected by RLS for >>> > non-superusers, at least as an option. >> I think that's a complete nonstarter. Aside from the fact that such a >> constraint will hav

Re: [HACKERS] UNION ALL on partitioned tables won't use indices.

2013-10-29 Thread Kyotaro HORIGUCHI
Hello, > Please add your patches to the currently-open CommitFest so that we > don't lose track of them. > > https://commitfest.postgresql.org/action/commitfest_view/open > > I'm not sure which approach to this problem is best, but I agree that > it is worth solving. Thank you, I've regsitered

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Craig Ringer
On 10/30/2013 10:50 AM, Tom Lane wrote: > Craig Ringer writes: >> > I'd kind of like to see FK constraints affected by RLS for >> > non-superusers, at least as an option. > I think that's a complete nonstarter. Aside from the fact that such a > constraint will have no definable semantics, even th

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Tom Lane
Craig Ringer writes: > I'd kind of like to see FK constraints affected by RLS for > non-superusers, at least as an option. I think that's a complete nonstarter. Aside from the fact that such a constraint will have no definable semantics, even the possibility that a constraint doesn't mean what i

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Craig Ringer
On 10/29/2013 11:21 PM, Kohei KaiGai wrote: > My vote is, system should keep referencial integrity as if RLS policy is > not configured. It is more fundamental stuff than RLS policy per user > basis. > I agree, and right now that is not how it works, causing some pretty confusing behaviour. --

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Craig Ringer
On 10/29/2013 10:01 PM, Tom Lane wrote: > As I recall, I've been saying since day one that row-level security cannot > sensibly coexist with foreign-key constraints, and I've been told that the > potential users of such a feature don't care. I'm glad to see somebody > else complaining. I'm concer

Re: [HACKERS] Something fishy happening on frogmouth

2013-10-29 Thread Andrew Dunstan
On 10/29/2013 03:47 PM, Andrew Dunstan wrote: On 10/29/2013 03:12 PM, Tom Lane wrote: It may not be unrelated that this machine was happy before commit d2aecae went in. I'll try a run with that reverted just to see if that's it. This is a 32 bit compiler on a 32 bit (virtual) machine,

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Leonardo Francalanci
> Hmm, you realise Alvaro is working on MinMax indexes in this release? > They are very efficient with regard to index inserts and specially > designed for use on large tables. > > Prior work by Heikki on Grouped Item Tuples was a way of reducing the > size of indexes, yet still allowing uniquene

Re: [HACKERS] Something fishy happening on frogmouth

2013-10-29 Thread Andrew Dunstan
On 10/29/2013 03:12 PM, Tom Lane wrote: The last two buildfarm runs on frogmouth have failed in initdb, like this: creating directory d:/mingw-bf/root/HEAD/pgsql.2492/src/test/regress/./tmp_check/data ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting def

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Simon Riggs
On 29 October 2013 07:53, Leonardo Francalanci wrote: > I don't see much interest in insert-efficient indexes. Hmm, you realise Alvaro is working on MinMax indexes in this release? They are very efficient with regard to index inserts and specially designed for use on large tables. Prior work by

[HACKERS] Something fishy happening on frogmouth

2013-10-29 Thread Tom Lane
The last two buildfarm runs on frogmouth have failed in initdb, like this: creating directory d:/mingw-bf/root/HEAD/pgsql.2492/src/test/regress/./tmp_check/data ... ok creating subdirectories ... ok selecting default max_connections ... 100 selecting default shared_buffers ... 128MB selecting dyn

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Tom Lane
Jeff Janes writes: > Robert removed the lmgr lock on the meta page by using a retry loop with > lightweight locks. I've outlined how to remove the heavyweight lock on the > bucket page as well, but it would require an on-disk change to the index so > that each page knows how far the bucket it is

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Jeff Janes
On Tue, Oct 29, 2013 at 8:16 AM, Tom Lane wrote: > Leonardo Francalanci writes: > >> Before getting too excited about some new academic index type, it's > worth > >> noting the sad state in which hash indexes have languished for years. > > > Aren't hash indexes in a poor state because they are n

Re: [HACKERS] logical changeset generation v6.2

2013-10-29 Thread Robert Haas
On Tue, Oct 29, 2013 at 11:43 AM, Andres Freund wrote: >> I think modifying GetNewRelFileNode() is attacking the problem from >> the wrong end. The point is that when a table is dropped, that fact >> can be communicated to the same machine machinery that's been tracking >> the CTID->CTID mappings

Re: [HACKERS] CLUSTER FREEZE

2013-10-29 Thread Robert Haas
On Tue, Oct 29, 2013 at 11:37 AM, Andres Freund wrote: > On 2013-10-29 11:29:24 -0400, Robert Haas wrote: >> On Tue, Oct 29, 2013 at 10:32 AM, Andres Freund >> wrote: >> > On 2013-10-25 09:26:29 -0400, Robert Haas wrote: >> >> > In any case, it's very far from obvious to me that CLUSTER ought >>

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Leonardo Francalanci
> I bet you've mis-diagnosed the problem.  Btrees don't have a problem > keeping up with 50m records; you're problem is that after a certain > point your page cache can't keep up with the pseudo-random i/o > patterns and you start seeing faults to storage. > [...]  This has nothing to do the btree

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Peter Geoghegan
On Tue, Oct 29, 2013 at 7:53 AM, Leonardo Francalanci wrote: > I don't see much interest in insert-efficient indexes. Presumably someone will get around to implementing a btree index insertion buffer one day. I think that would be a particularly compelling optimization for us, because we could av

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Merlin Moncure
On Tue, Oct 29, 2013 at 10:49 AM, Leonardo Francalanci wrote: >> Another point to add: I don't really see btree as a barrier to >> performance for most of the problems I face. The real barriers to >> database performance are storage, contention, and query planning. > > Ehm that's true for regular

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Claudio Freire
On Tue, Oct 29, 2013 at 1:10 PM, Peter Geoghegan wrote: > On Tue, Oct 29, 2013 at 7:53 AM, Leonardo Francalanci > wrote: > > I don't see much interest in insert-efficient indexes. > > Presumably someone will get around to implementing a btree index > insertion buffer one day. I think that would

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread David Johnston
Tom Lane-2 wrote > Craig Ringer < > craig@ > > writes: >> During my testing of Kohei KaiGai's row-security patches I've been >> looking into how foreign keys should be and are handled. There are some >> interesting wrinkles around FK cascades, the rights under which FK >> checks execute, and abou

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Leonardo Francalanci
> They should, in theory, be faster than btrees -- O(1) not O(log N) page > fetches per lookup.  In practice they don't seem to be faster, and > nobody's bothered to find out exactly why.  Again, this isn't a terribly > encouraging precedent for implementing some other index type that's > supposed

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Leonardo Francalanci
> Another point to add: I don't really see btree as a barrier to > performance for most of the problems I face.  The real barriers to > database performance are storage, contention, and query planning. Ehm that's true for regular OLTP stuff, which I understand is what most (95%?) of people use/ne

Re: [HACKERS] logical changeset generation v6.2

2013-10-29 Thread Andres Freund
On 2013-10-29 11:28:44 -0400, Robert Haas wrote: > On Tue, Oct 29, 2013 at 10:47 AM, Andres Freund > wrote: > > On 2013-10-28 11:54:31 -0400, Robert Haas wrote: > >> > There's one snag I currently can see, namely that we actually need to > >> > prevent that a formerly dropped relfilenode is getti

Re: [HACKERS] CLUSTER FREEZE

2013-10-29 Thread Andres Freund
On 2013-10-29 11:29:24 -0400, Robert Haas wrote: > On Tue, Oct 29, 2013 at 10:32 AM, Andres Freund > wrote: > > On 2013-10-25 09:26:29 -0400, Robert Haas wrote: > >> > In any case, it's very far from obvious to me that CLUSTER ought > >> > to throw away information by default, which is what you'r

Re: [HACKERS] CLUSTER FREEZE

2013-10-29 Thread Robert Haas
On Tue, Oct 29, 2013 at 10:32 AM, Andres Freund wrote: > On 2013-10-25 09:26:29 -0400, Robert Haas wrote: >> > In any case, it's very far from obvious to me that CLUSTER ought >> > to throw away information by default, which is what you're proposing. >> >> I find it odd to referring to this as thr

Re: [HACKERS] logical changeset generation v6.2

2013-10-29 Thread Robert Haas
On Tue, Oct 29, 2013 at 10:47 AM, Andres Freund wrote: > On 2013-10-28 11:54:31 -0400, Robert Haas wrote: >> > There's one snag I currently can see, namely that we actually need to >> > prevent that a formerly dropped relfilenode is getting reused. Not >> > entirely sure what the best way for that

Re: [HACKERS] stats for network traffic WIP

2013-10-29 Thread Nigel Heron
> > So, for now, the counters only track sockets created from an inbound > (client to server) connection. here's v3 of the patch (rebase and cleanup). -nigel. *** a/src/backend/catalog/system_views.sql --- b/src/backend/catalog/system_views.sql *** *** 586,592 CREATE VIEW pg_stat

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Kohei KaiGai
2013/10/29 Tom Lane : > Craig Ringer writes: >> During my testing of Kohei KaiGai's row-security patches I've been >> looking into how foreign keys should be and are handled. There are some >> interesting wrinkles around FK cascades, the rights under which FK >> checks execute, and about the consi

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Tom Lane
Leonardo Francalanci writes: >> Before getting too excited about some new academic index type, it's worth >> noting the sad state in which hash indexes have languished for years. > Aren't hash indexes in a poor state because they are not faster than btree in > every condition? They should, in t

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread k...@rice.edu
On Tue, Oct 29, 2013 at 02:53:37PM +, Leonardo Francalanci wrote: > > Before getting too excited about some new academic index type, it's worth > > noting the sad state in which hash indexes have languished for years. > > Nobody's bothered to add WAL support, let alone do any other real work >

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Alvaro Herrera
Leonardo Francalanci wrote: > > Before getting too excited about some new academic index type, it's worth > > noting the sad state in which hash indexes have languished for years. > > Nobody's bothered to add WAL support, let alone do any other real work > > on them.  The non-btree index types that

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Leonardo Francalanci
> Before getting too excited about some new academic index type, it's worth > noting the sad state in which hash indexes have languished for years. > Nobody's bothered to add WAL support, let alone do any other real work > on them.  The non-btree index types that have been getting love are the > on

Re: [HACKERS] logical changeset generation v6.2

2013-10-29 Thread Andres Freund
On 2013-10-28 11:54:31 -0400, Robert Haas wrote: > > There's one snag I currently can see, namely that we actually need to > > prevent that a formerly dropped relfilenode is getting reused. Not > > entirely sure what the best way for that is. > > I'm not sure in detail, but it seems to me that thi

Re: [HACKERS] CLUSTER FREEZE

2013-10-29 Thread Andres Freund
On 2013-10-25 09:26:29 -0400, Robert Haas wrote: > > In any case, it's very far from obvious to me that CLUSTER ought > > to throw away information by default, which is what you're proposing. > > I find it odd to referring to this as throwing away information. I > know that you have a general con

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Merlin Moncure
On Tue, Oct 29, 2013 at 2:53 AM, Leonardo Francalanci wrote: > Hi, > > > I don't see much interest in insert-efficient indexes. These are the ones > I've found: > > - LSM-tree (used by Cassandra and SQLite4?) > - Y-Tree > (http://www.bossconsulting.com/oracle_dba/white_papers/DW%20in%20oracle/P2

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Tom Lane
Craig Ringer writes: > On 10/29/2013 03:53 PM, Leonardo Francalanci wrote: >> 5) something else??? > Quite likely nobody has had the enthusiasm and interest to implement a > viable, quality implementation and stick with it long enough to get it > committed. > There are a great many good ideas fo

[HACKERS] Creating partial index on a relation

2013-10-29 Thread naman.iitb
Hello I am doing a small project in Postgress where i have to achieve the following: Suppose i know the index name(lets say index1) and the relation(table1) on which partial index to has to be build. I was looking through the code and found that IndexStmt-->whereClause is the one that i need to

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Tom Lane
Craig Ringer writes: > During my testing of Kohei KaiGai's row-security patches I've been > looking into how foreign keys should be and are handled. There are some > interesting wrinkles around FK cascades, the rights under which FK > checks execute, and about the consistency effects of changing o

Re: [HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Craig Ringer
On 10/29/2013 03:53 PM, Leonardo Francalanci wrote: > 5) something else??? Quite likely nobody has had the enthusiasm and interest to implement a viable, quality implementation and stick with it long enough to get it committed. There are a great many good ideas for improvements to Pg that just do

Re: [HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Craig Ringer
On 10/29/2013 04:09 PM, Craig Ringer wrote: > Problem is, that won't necessarily happen, because the FK check is run > with the rights of the table owner. Some further reading suggests that another vendor's implementation ignores row security policy for foreign key constraint checks. So FK constra

[HACKERS] How should row-security affects ON UPDATE RESTRICT / CASCADE ?

2013-10-29 Thread Craig Ringer
During my testing of Kohei KaiGai's row-security patches I've been looking into how foreign keys should be and are handled. There are some interesting wrinkles around FK cascades, the rights under which FK checks execute, and about the consistency effects of changing or applying an RLS policy. It

[HACKERS] Fast insertion indexes: why no developments

2013-10-29 Thread Leonardo Francalanci
Hi, I don't see much interest in insert-efficient indexes. These are the ones I've found: - LSM-tree (used by Cassandra and SQLite4?) - Y-Tree (http://www.bossconsulting.com/oracle_dba/white_papers/DW%20in%20oracle/P23%20(ytree%20index%20structure%20for%20DWs).pdf ) - Fractal indexes (TokuDB,

Re: [HACKERS] PostgreSQL Service on Windows does not start. ~ "is not a valid Win32 application"

2013-10-29 Thread Naoya Anzai
Hi Sandeep > I think, you should change the subject line to "Unquoted service path > containing space is vulnerable and can be exploited on Windows" to get the > attention.. :) Thank you for advice! I'll try to post to pgsql-bugs again. > BTW, in your case, the file "Program" should be an exe