Re: [HACKERS] Rowcounts marked by create_foreignscan_path()

2014-03-03 Thread Etsuro Fujita
(2014/02/18 12:37), Tom Lane wrote: > Etsuro Fujita writes: >> (2014/02/18 12:03), Tom Lane wrote: >>> The calling FDW is supposed to do that; note the header comment. > >> However, ISTM postgresGetForeignPaths() doesn't work like >> that. It uses the same rowcount for all paths of a same parame

Re: [HACKERS] [PATCH] Use MAP_HUGETLB where supported (v3)

2014-03-03 Thread Christian Kruse
Hi, > >Attached is a patch with the updated documentation (now uses > >consistently huge pages) as well as a renamed GUC, consistent wording > >(always use huge pages) as well as renamed variables. > > Hmm, I wonder if that could now be misunderstood to have something to do > with the PostgreSQL

Re: [HACKERS] [PATCH] Use MAP_HUGETLB where supported (v3)

2014-03-03 Thread Christian Kruse
Hi, On 28/02/14 17:58, Peter Geoghegan wrote: > On Fri, Feb 28, 2014 at 9:43 AM, Heikki Linnakangas > wrote: > > Hmm, I wonder if that could now be misunderstood to have something to do > > with the PostgreSQL page size? Maybe add the word "memory" or "operating > > system" in the first sentence

Re: [HACKERS] UNION ALL on partitioned tables won't use indices.

2014-03-03 Thread Kyotaro HORIGUCHI
Hello, > > Yes, the old dumped version of typ2 patch did so. It flattened > > appendrel tree for the query prpoerly. Let me hear the reson you > > prefer to do so. > > Having reviewed my upthread reasoning for preferring one of those two > approaches over the other, it's a weak preference. They

Re: [HACKERS] Get more from indices.

2014-03-03 Thread Kyotaro HORIGUCHI
I marked this patch as 'Ready for Committer' by myself according to the following discussion. Thanks. > At Tue, 25 Feb 2014 16:35:55 -0500, Robert Haas wrote > > On Tue, Feb 25, 2014 at 3:36 AM, Kyotaro HORIGUCHI > > wrote: > > > May I mark this patch as "Ready for Committer" by myself since > >

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-03 Thread Andres Freund
On 2014-02-28 20:55:20 +0530, Amit Kapila wrote: > On Thu, Feb 27, 2014 at 4:14 PM, Christian Kruse > >> I wouldn't be inclined to dump the whole tuple under any > >> circumstances. That could be a lot more data than what you want > >> dumped in your log. The PK could already be somewhat unreason

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-03 Thread Andres Freund
On 2014-03-01 13:29:18 +0530, Amit Kapila wrote: > With new patch, the message while updating locked rows will be displayed > as below: > > LOG: process 364 still waiting for ShareLock on transaction 678 after > 1014.000ms > CONTEXT: while attempting to lock tuple (0,2) with values (2) in relati

Re: [HACKERS] GSoC proposal

2014-03-03 Thread Albe Laurenz
> I'm applying for GSoC 2014 with Postgresql and would appreciate your comments > on my proposal > (attached). I'm looking for technical corrections/comments and your opinions > on the project's > viability. In particular, if the community has doubts about its usefulness, I > would start working

Re: [HACKERS] gaussian distribution pgbench

2014-03-03 Thread KONDO Mitsumasa
(2014/03/03 16:51), Fabien COELHO wrote:>>>\setrandom foo 1 10 [uniform] >>>\setrandom foo 1 :size gaussian 3.6 >>>\setrandom foo 1 100 exponential 7.2 >> It's good design. I think it will become more low overhead at part of parsing >> in pgbench, because comparison of strings will be

Re: [HACKERS] Review: Patch FORCE_NULL option for copy COPY in CSV mode

2014-03-03 Thread Michael Paquier
On Mon, Mar 3, 2014 at 1:13 PM, Andrew Dunstan wrote: >>> That difference actually made the file_fdw regression results plain >>> wrong, >>> in my view, in that they expected a quoted empty string to be turned to >>> null >>> even when the null string was something else. >>> >>> I've adjusted this

Re: [HACKERS] heapgetpage() and ->takenDuringRecovery

2014-03-03 Thread Robert Haas
On Sun, Mar 2, 2014 at 8:39 AM, Andres Freund wrote: > While reading around which references to SnapshotData's members exist, I > once more came about the following tidbit in heapgetpage(): > /* > * If the all-visible flag indicates that all tuples on the page are > * vis

Re: [HACKERS] Defining macro LSNOID for pg_lsn in pg_type.h

2014-03-03 Thread Robert Haas
On Thu, Feb 27, 2014 at 8:12 AM, Michael Paquier wrote: > When working on the datatype pg_lsn, we actually did not create a > define macro for its oid in pg_type.h and this could be useful for > extension developers. The simple patch attached corrects that by > naming this macro LSNOID. Thanks, c

Re: [HACKERS] heapgetpage() and ->takenDuringRecovery

2014-03-03 Thread Andres Freund
On 2014-03-03 06:57:00 -0500, Robert Haas wrote: > On Sun, Mar 2, 2014 at 8:39 AM, Andres Freund wrote: > > While reading around which references to SnapshotData's members exist, I > > once more came about the following tidbit in heapgetpage(): > > /* > > * If the all-visible flag

Re: [HACKERS] Dump pageinspect to 1.2: page_header with pg_lsn datatype

2014-03-03 Thread Robert Haas
On Thu, Feb 27, 2014 at 8:05 AM, Michael Paquier wrote: > On Wed, Feb 26, 2014 at 5:45 AM, Alvaro Herrera > wrote: >> Robert Haas escribió: >>> On Mon, Feb 24, 2014 at 9:34 AM, Alvaro Herrera >>> wrote: >>> > Yeah, erroring out seems good enough. Particularly if you add a hint >>> > saying "ple

Re: [HACKERS] heapgetpage() and ->takenDuringRecovery

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 7:07 AM, Andres Freund wrote: >> I don't think there's any reason to believe that lazy_scan_heap() can >> only hit pages that are empty or have just been defragged. Suppose >> that there's a tuple on the page which was recently inserted; the >> inserting transaction has com

Re: [HACKERS] Defining macro LSNOID for pg_lsn in pg_type.h

2014-03-03 Thread Michael Paquier
On Mon, Mar 3, 2014 at 9:06 PM, Robert Haas wrote: > On Thu, Feb 27, 2014 at 8:12 AM, Michael Paquier > wrote: >> When working on the datatype pg_lsn, we actually did not create a >> define macro for its oid in pg_type.h and this could be useful for >> extension developers. The simple patch attac

Re: [HACKERS] VACUUM FULL/CLUSTER doesn't update pg_class's pg_class.relfrozenxid

2014-03-03 Thread Robert Haas
On Thu, Feb 27, 2014 at 1:06 PM, Andres Freund wrote: > As Robert previously complained a database wide VACUUM FULL now (as of > 3cff1879f8d03) reliably increases the relfrozenxid for all tables but > pg_class itself. That's a bit sad because it means doing a VACUUM FULL > won't help in a anti-wra

[HACKERS] Re: VACUUM FULL/CLUSTER doesn't update pg_class's pg_class.relfrozenxid

2014-03-03 Thread Andres Freund
On 2014-03-03 07:52:23 -0500, Robert Haas wrote: > And I think that's a pretty worthwhile thing to do, because we get > periodic reports from people who have run VACUUM FULL on a database in > danger of wraparound and then wondered why it did not fix the problem. > The previously-mentioned commit

Re: [HACKERS] Securing "make check" (CVE-2014-0067)

2014-03-03 Thread Andrew Dunstan
On 03/03/2014 02:00 AM, Tom Lane wrote: Josh Berkus writes: The only way I can see this being of real use to an attacker is if they could use this exploit to create a wormed version of PostgresQL on the target build system. Is that possible? It's theoretically possible, since having broken i

Re: [HACKERS] heapgetpage() and ->takenDuringRecovery

2014-03-03 Thread Andres Freund
On 2014-03-03 06:57:00 -0500, Robert Haas wrote: > On Sun, Mar 2, 2014 at 8:39 AM, Andres Freund wrote: > > I don't think this is neccessary >= 9.2. The are two only "interestings" > > place > > where PD_ALL_VISIBLE is set: > > a) lazy_vacuum_page() where a xl_heap_clean is logged *before* > >

Re: [HACKERS] Triggers on foreign tables

2014-03-03 Thread Kohei KaiGai
I tried to check the latest (v8) patch again, then could not find problem in your design change from the v7. As Noah pointed out, it uses per query-depth tuplestore being released on AfterTriggerEndSubXact. So, may I mark it as "ready for committer"? 2014-03-03 15:48 GMT+09:00 Ronan Dunklau : > H

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-03-03 Thread Heikki Linnakangas
On 02/16/2014 01:51 PM, Amit Kapila wrote: On Wed, Feb 5, 2014 at 5:29 PM, Heikki Linnakangas wrote: >I'm pretty sure the overhead of that would be negligible, so we could always >enable it. There are certainly a lot of scenarios where prefix/suffix >detection alone wouldn't help, but so what.

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-03-03 Thread Andres Freund
On 2014-03-03 16:27:05 +0200, Heikki Linnakangas wrote: > Thanks. I have to agree with Robert though that using the pglz encoding when > we're just checking for a common prefix/suffix is a pretty crappy way of > going about it [1]. > > As the patch stands, it includes the NULL bitmap when checking

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Merlin Moncure
On Fri, Feb 28, 2014 at 2:01 PM, Andres Freund wrote: > On 2014-02-28 14:45:29 -0500, Andrew Dunstan wrote: >> Well, the jsonb portion of this is arguably the most ready, certainly it's >> had a lot more on-list review. > > Having crossread both patches I tend to agree with this. I don't think > i

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Andres Freund
On 2014-03-03 08:57:59 -0600, Merlin Moncure wrote: > On Fri, Feb 28, 2014 at 2:01 PM, Andres Freund wrote: > > On 2014-02-28 14:45:29 -0500, Andrew Dunstan wrote: > >> Well, the jsonb portion of this is arguably the most ready, certainly it's > >> had a lot more on-list review. > > > > Having cro

Re: [HACKERS] Triggers on foreign tables

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 11:10:30PM +0900, Kohei KaiGai wrote: > I tried to check the latest (v8) patch again, then could not find > problem in your design change from the v7. > As Noah pointed out, it uses per query-depth tuplestore being released > on AfterTriggerEndSubXact. > > So, may I mark it

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Oleg Bartunov
Andres, you can always look at our development repository: https://github.com/feodor/postgres/tree/hstore - hstore only, https://github.com/feodor/postgres/tree/jsonb_and_hstore - hstore with jsonb Since we were concentrated on the jsonb_and_hstore branch we usually wait Andrew, who publish patch

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: > Removing SELECT privilege while running a SELECT would be a different > matter. This is all a matter of definition; we can make up any rules > we like. Doing so is IMHO a separate patch and not something to hold > up the main patch. So I thin

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Andres Freund
Hi Oleg, On 2014-03-03 19:17:12 +0400, Oleg Bartunov wrote: > Since we were concentrated on the jsonb_and_hstore branch we usually > wait Andrew, who publish patch. You last issues were addressed in > both branches. I'll try to have look sometime soon. > We are not native-english and may not we

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Oleg Bartunov
On Mon, Mar 3, 2014 at 7:22 PM, Andres Freund wrote: > Hi Oleg, > > On 2014-03-03 19:17:12 +0400, Oleg Bartunov wrote: >> Since we were concentrated on the jsonb_and_hstore branch we usually >> wait Andrew, who publish patch. You last issues were addressed in >> both branches. > > I'll try to hav

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-03-03 Thread Robert Haas
On Sat, Mar 1, 2014 at 9:04 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> I don't see that parallelizing Append is any easier than any other >> problem in this space. There's no parallel I/O facility, so you need >> a background worker per append branch to wait on I/O

Re: [HACKERS] commit fest status and release timeline

2014-03-03 Thread Tomas Vondra
On 1.3.2014 18:01, Peter Eisentraut wrote: > Status Summary. Needs Review: 36, Waiting on Author: 7, Ready for > Committer: 16, Committed: 43, Returned with Feedback: 8, Rejected: > 4. Total: 114. > > We're still on track to achieve about 50% committed patches, which > would be similar to the pre

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-03-03 Thread Andres Freund
On 2014-03-03 10:35:03 -0500, Robert Haas wrote: > On Mon, Mar 3, 2014 at 9:57 AM, Andres Freund wrote: > > Hm, I think all it needs to do disable delta encoding if > > need_tuple_data (which is dependent on wal_level=logical). > > Why does it need to do that? The logical decoding stuff should b

Re: [HACKERS] Request improve pg_stat_statements module

2014-03-03 Thread Robert Haas
On Fri, Feb 28, 2014 at 8:01 AM, Michael Paquier wrote: > Thanks for your patch! > > On Fri, Feb 28, 2014 at 4:18 PM, wrote: >> I patched to add one column in pg_stat_statements module. >> and sent to author but >> I need a last time of query, because I want to analyse order by recent time. > Hm

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 9:57 AM, Andres Freund wrote: > Hm, I think all it needs to do disable delta encoding if > need_tuple_data (which is dependent on wal_level=logical). Why does it need to do that? The logical decoding stuff should be able to reverse out the delta encoding. -- Robert Haas

Re: [HACKERS] pg_dump reporing version of server & pg_dump as comments in the output

2014-03-03 Thread Euler Taveira
On 27-02-2014 21:10, Wang, Jing wrote: > Using pg_dump can dump the data into the file with format set to be > 'c','t' or plain text. In the existing version the version of server & > pg_dump is already there when the format of file is 'c' or 't'. And even > for the plain text format file the versi

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-03-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Sat, Mar 1, 2014 at 9:04 PM, Stephen Frost wrote: > > Erm, my thought was to use a select() loop which sends out I/O requests > > and then loops around waiting to see who finishes it. It doesn't > > parallelize the CPU cost of getting the rows bac

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 15:19, Robert Haas wrote: > On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: > What I'm > really concerned about is whether there are other things like the > SnapshotNow issues that can cause stuff to halt and catch fire. I > don't know whether there are or are not, but that'

Re: [HACKERS] heapgetpage() and ->takenDuringRecovery

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 8:33 AM, Andres Freund wrote: > On 2014-03-03 06:57:00 -0500, Robert Haas wrote: >> On Sun, Mar 2, 2014 at 8:39 AM, Andres Freund wrote: >> > I don't think this is neccessary >= 9.2. The are two only "interestings" >> > place >> > where PD_ALL_VISIBLE is set: >> > a) lazy_

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 10:38 AM, Andres Freund wrote: > On 2014-03-03 10:35:03 -0500, Robert Haas wrote: >> On Mon, Mar 3, 2014 at 9:57 AM, Andres Freund wrote: >> > Hm, I think all it needs to do disable delta encoding if >> > need_tuple_data (which is dependent on wal_level=logical). >> >> Why

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Simon Riggs writes: > On 3 March 2014 15:19, Robert Haas wrote: >> What I'm >> really concerned about is whether there are other things like the >> SnapshotNow issues that can cause stuff to halt and catch fire. I >> don't know whether there are or are not, but that's my concern. > Of course it

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 10:43 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Sat, Mar 1, 2014 at 9:04 PM, Stephen Frost wrote: >> > Erm, my thought was to use a select() loop which sends out I/O requests >> > and then loops around waiting to see who finishes it. It

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs wrote: > v20 includes slightly re-ordered checks in GetLockLevel, plus more > detailed comments on each group of subcommands. > > Also corrects grammar as noted by Vik. > > Plus adds an example of usage to the docs. This patch contains a one line change

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-03-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > > http://www.postgresql.org/message-id/20131104032604.gb2...@tamriel.snowman.net > > Huh, somehow I can't remember reading that... but I didn't think I had > missed any posts, either. Evidently I did. You and everyone else- you'll note it got exactl

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-03-03 Thread Robert Haas
On Sun, Mar 2, 2014 at 2:38 PM, Tan Tran wrote: > 2. Proposal > As a GSoC student, I will implement WAL recovery of hash indexes using the > other index types' WAL code as a guide. Roughly, I will: > - Devise a way to store and retrieve hashing data within the XLog data > structures. > - In the ex

Re: [HACKERS] GiST support for inet datatypes

2014-03-03 Thread Robert Haas
On Thu, Feb 27, 2014 at 12:30 PM, Tom Lane wrote: > Florian Pflug writes: >> On Feb27, 2014, at 17:56 , Tom Lane wrote: >>> That's not a bug, it's a feature, for much the same reasons that pg_dump >>> tries to minimize explicit schema-qualification. > >> I fail to see the value in this for opcla

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 15:53, Tom Lane wrote: > Simon Riggs writes: >> On 3 March 2014 15:19, Robert Haas wrote: >>> What I'm >>> really concerned about is whether there are other things like the >>> SnapshotNow issues that can cause stuff to halt and catch fire. I >>> don't know whether there are or

[HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Fabrízio de Royes Mello
Hi all, Is the TODO item "make an unlogged table logged" [1] a good GSoC project? Regards, [1] http://www.postgresql.org/message-id/aanlktinenzbrxdcwohkqbba2bhubfy8_c5jwrxlod...@mail.gmail.com -- Fabrízio de Royes Mello Consultoria/Coaching PostgreSQL >> Timbira: http://www.timbira.com.br >> B

Re: [HACKERS] commit fest status and release timeline

2014-03-03 Thread Magnus Hagander
On Mon, Mar 3, 2014 at 4:34 PM, Tomas Vondra wrote: > On 1.3.2014 18:01, Peter Eisentraut wrote: > > Status Summary. Needs Review: 36, Waiting on Author: 7, Ready for > > Committer: 16, Committed: 43, Returned with Feedback: 8, Rejected: > > 4. Total: 114. > > > > We're still on track to achieve

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 16:06, Robert Haas wrote: > On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs wrote: >> v20 includes slightly re-ordered checks in GetLockLevel, plus more >> detailed comments on each group of subcommands. >> >> Also corrects grammar as noted by Vik. >> >> Plus adds an example of usage

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 11:29 AM, Simon Riggs wrote: > On 3 March 2014 16:06, Robert Haas wrote: >> On Sun, Mar 2, 2014 at 4:50 AM, Simon Riggs wrote: >>> v20 includes slightly re-ordered checks in GetLockLevel, plus more >>> detailed comments on each group of subcommands. >>> >>> Also corrects g

Re: [HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 11:28 AM, Fabrízio de Royes Mello wrote: > Is the TODO item "make an unlogged table logged" [1] a good GSoC project? I'm pretty sure we found some problems in that design that we couldn't figure out how to solve. I don't have a pointer to the relevant -hackers discussion o

Re: [HACKERS] GiST support for inet datatypes

2014-03-03 Thread Tom Lane
Robert Haas writes: > On Thu, Feb 27, 2014 at 12:30 PM, Tom Lane wrote: >> The value in it is roughly the same as the reason we don't include a >> version number when dumping CREATE EXTENSION. If you had a default >> opclass in the source database, you probably want a default opclass >> in the d

Re: [HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Mon, Mar 3, 2014 at 11:28 AM, Fabrízio de Royes Mello > wrote: > > Is the TODO item "make an unlogged table logged" [1] a good GSoC project? > > I'm pretty sure we found some problems in that design that we couldn't > figure out how to solve. I d

Re: [HACKERS] UNION ALL on partitioned tables won't use indices.

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 07:01:09PM +0900, Kyotaro HORIGUCHI wrote: > > > Yes, the old dumped version of typ2 patch did so. It flattened > > > appendrel tree for the query prpoerly. Let me hear the reson you > > > prefer to do so. > > > > Having reviewed my upthread reasoning for preferring one of

Re: [HACKERS] patch: option --if-exists for pg_dump

2014-03-03 Thread Alvaro Herrera
Pavel Stehule escribió: > This patch has redesigned implementation --if-exists for pg_dumpall. Now it > is not propagated to pg_dump, but used on pg_dumpall level. Seems sane, thanks. BTW after this patch, I still don't see an error-free output from restoring a database on top of itself. One pr

Re: [HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Tom Lane
Stephen Frost writes: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Mon, Mar 3, 2014 at 11:28 AM, Fabrízio de Royes Mello >> wrote: >>> Is the TODO item "make an unlogged table logged" [1] a good GSoC project? >> I'm pretty sure we found some problems in that design that we couldn't >> fi

Re: [HACKERS] GSoC on WAL-logging hash indexes

2014-03-03 Thread Tan Tran
Thanks for alerting me to your previous idea. While I don't know enough about Postgresql internals to judge its merits yet, I'll write some pseudocode based on it in my proposal; and I'll relegate it to a "reach" proposal alongside a more straightforward one. Tan On Mon, Mar 3, 2014 at 8:12 AM, R

Re: [HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Hannu Krosing
On 03/03/2014 05:22 PM, Tom Lane wrote: > Stephen Frost writes: ... >> ISTR the discussion going something along the lines of "we'd have to WAL >> log the entire table to do that, and if we have to do that, what's the >> point?". > IIRC, the reason you'd have to do that is to make the table conten

Re: [HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Andres Freund
On 2014-03-03 12:08:26 -0500, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > On Mon, Mar 3, 2014 at 11:28 AM, Fabrízio de Royes Mello > > wrote: > > > Is the TODO item "make an unlogged table logged" [1] a good GSoC project? > > > > I'm pretty sure we found some problems

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: > On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: > > Removing SELECT privilege while running a SELECT would be a different > > matter. This is all a matter of definition; we can make up any rules > > we like. Doing so is IMHO a sep

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 03:43:46PM +, Simon Riggs wrote: > The question is are there any specific areas of concern here? If not, > then we commit because we've done a lot of work on it and at the > moment the balance is high benefit to users against a non-specific > feeling of risk. > > @Noah

Re: [HACKERS] UNION ALL on partitioned tables won't use indices.

2014-03-03 Thread Tom Lane
Noah Misch writes: > If you are convinced that a separate flattening pass is best, that suffices > for me at this stage. Please submit the patch you have in mind, incorporating > any improvements from the v7 patch that are relevant to both approaches. I went back and re-read the original message

Re: [HACKERS] [PATCH] Use MAP_HUGETLB where supported (v3)

2014-03-03 Thread Heikki Linnakangas
On 03/03/2014 11:34 AM, Christian Kruse wrote: Hi, Attached is a patch with the updated documentation (now uses consistently huge pages) as well as a renamed GUC, consistent wording (always use huge pages) as well as renamed variables. Hmm, I wonder if that could now be misunderstood to have

Re: [HACKERS] integrate pg_upgrade analyze_new_cluster.sh into vacuumdb

2014-03-03 Thread Kevin Grittner
Jeff Janes wrote: > But I do wonder what experience people have with the 3 stage > process, how useful is it empirically?  If you can't open the > database for general use until the 3rd phase is done, then you > would just jump to doing that stage, rather than working through > all 3 of them.  If

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Simon Riggs
On 3 March 2014 18:57, Noah Misch wrote: > On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: >> On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: >> > Removing SELECT privilege while running a SELECT would be a different >> > matter. This is all a matter of definition; we can make u

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Gavin Flower
On 04/03/14 04:25, Oleg Bartunov wrote: On Mon, Mar 3, 2014 at 7:22 PM, Andres Freund wrote: [...] PS: Not a native speaker either... That's explain all :) [...] I AM a native English speaker born in England - though if you read some of my postings where I've been particularly careless, y

Re: [HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Peter Geoghegan
On Mon, Mar 3, 2014 at 8:28 AM, Fabrízio de Royes Mello wrote: > Is the TODO item "make an unlogged table logged" [1] a good GSoC project? Another interesting project around unlogged tables would be to make it possible to have unlogged indexes on fully-logged tables. That is something that there

Re: [HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Andres Freund
On 2014-03-03 12:44:26 -0800, Peter Geoghegan wrote: > On Mon, Mar 3, 2014 at 8:28 AM, Fabrízio de Royes Mello > wrote: > > Is the TODO item "make an unlogged table logged" [1] a good GSoC project? > > Another interesting project around unlogged tables would be to make it > possible to have unlog

Re: [HACKERS] Changeset Extraction v7.9

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 11:26 AM, Andres Freund wrote: > On 2014-02-27 17:56:08 +0100, Andres Freund wrote: >> * do we modify struct SnapshotData to be polymorphic based on some tag >> or move comments there? > > I tried that, and it got far to invasive. So I've updated the relevant > comment in

[HACKERS] building pgadmin4

2014-03-03 Thread Willy-Bas Loos
Hi, I'm trying to build pgadmin4, out of curiosity. I'm on a ubuntu 13.10 desktop vm. I added qt webkitwidgets, and now I run into the next error, which doesn't seem to make much sense: wbloos2@vm1:~/pgadmin4/runtime$ qmake Project MESSAGE: Building for QT5+... Project ERROR: Unknown module(s) in

Re: [HACKERS] building pgadmin4

2014-03-03 Thread Robert Haas
On Mon, Mar 3, 2014 at 4:53 PM, Willy-Bas Loos wrote: > I'm trying to build pgadmin4, out of curiosity. > I'm on a ubuntu 13.10 desktop vm. > I added qt webkitwidgets, and now I run into the next error, which doesn't > seem to make much sense: > wbloos2@vm1:~/pgadmin4/runtime$ qmake > Project MESS

Re: [HACKERS] plpgsql.warn_shadow

2014-03-03 Thread Joel Jacobson
On Wed, Jan 15, 2014 at 1:34 AM, Marko Tiikkaja wrote: > Hi all! > > It's me again, trying to find a solution to the most common mistakes I make. > This time it's accidental shadowing of variables, especially input > variables. I've wasted several hours banging my head against the wall while > sh

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-03-03 Thread Stephen Frost
KaiGai, * Stephen Frost (sfr...@snowman.net) wrote: > And I'm still unconvinced of this approach and worry that it's going to > break more often than it works. That's my 2c on it, but I won't get in > the way if someone else wants to step up and support it. Alright, having heard from Robert (tha

Re: [HACKERS] building pgadmin4

2014-03-03 Thread Willy-Bas Loos
> > pgAdmin is off-topic for this mailing list. > so sorry, i misread the adress in the readme file cheers, WBL -- "Quality comes from focus and clarity of purpose" -- Mark Shuttleworth

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 07:19:45PM +, Simon Riggs wrote: > On 3 March 2014 18:57, Noah Misch wrote: > > On Mon, Mar 03, 2014 at 10:19:55AM -0500, Robert Haas wrote: > >> On Thu, Feb 27, 2014 at 3:12 AM, Simon Riggs wrote: > >> > Removing SELECT privilege while running a SELECT would be a diff

Re: [HACKERS] Changeset Extraction v7.9

2014-03-03 Thread Andres Freund
Hi Robert, Everyone! On 2014-03-03 16:48:15 -0500, Robert Haas wrote: > OK, I've committed the 0001 patch, which is the core of this feature, > with a bit of minor additional hacking. Many, many, thanks! > I'm sure there are some problems here yet and some things that people > will want fixed, a

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-03-03 Thread Kouhei Kaigai
> > As I mentioned > > up-thread, I'd really like to see FDW join push-down, FDW aggregate > > push-down, parallel query execution, and parallel remote-FDW execution > > and I don't see this CustomScan approach as the right answer to any of > > those. > > In accordance with the above, what I'd lik

Re: [HACKERS] plpgsql.warn_shadow

2014-03-03 Thread Tom Lane
Joel Jacobson writes: > I strongly think it should be made an error, because it is most > certainly an error, and even if it's not, it's at least bad coding > style and the code should be fixed anyway, or if one is lazy, turn > this off in the config file and make it a warning instead. You're rea

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Noah Misch writes: > Just to be clear, that list is not a commentary on the particular patch at > hand. Those are merely the kinds of regressions to look for in a patch > affecting this area of the code. A complaint on pgsql-bugs just now reminded me of a specific area that needs to be looked at

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Peter Geoghegan
On Fri, Feb 28, 2014 at 2:12 PM, Peter Geoghegan wrote: > In order to make a rational decision to do the work incrementally, we > need to know what we're putting off until 9.5. AFAICT, we have these > operator classes that work fine with jsonb for the purposes of > hstore-style indexing (the hstor

Re: [HACKERS] Securing "make check" (CVE-2014-0067)

2014-03-03 Thread Noah Misch
On Mon, Mar 03, 2014 at 01:29:00AM -0500, Tom Lane wrote: > Noah Misch writes: > > Concerning the immediate fix for non-Windows systems, does any modern system > > ignore modes of Unix domain sockets? It appears to be a long-fixed problem: > > What I was envisioning was that we'd be relying on t

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Josh Berkus
On 03/03/2014 04:50 PM, Peter Geoghegan wrote: > I understand that there are ambitious plans for a VODKA-am that will > support indexing operations on nested structures that are a lot more > advanced than those enabled by the hstore operator classes included in > these patches. However, surely thes

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Peter Geoghegan
On Mon, Mar 3, 2014 at 4:57 PM, Josh Berkus wrote: > On 03/03/2014 04:50 PM, Peter Geoghegan wrote: >> I understand that there are ambitious plans for a VODKA-am that will >> support indexing operations on nested structures that are a lot more >> advanced than those enabled by the hstore operator

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Andres Freund
On 2014-03-03 19:15:27 -0500, Tom Lane wrote: > Noah Misch writes: > > Just to be clear, that list is not a commentary on the particular patch at > > hand. Those are merely the kinds of regressions to look for in a patch > > affecting this area of the code. > > A complaint on pgsql-bugs just now

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Josh Berkus
On 03/03/2014 05:07 PM, Peter Geoghegan wrote: >> Primary value is that in theory the hstore2 opclasses are available >> *now*, as opposed to a year from now. > > Well, yes, that's right. Although we cannot assume that VODKA will get > into 9.5 - it's a big project. Nor is it obvious to me that a

Re: [HACKERS] Securing "make check" (CVE-2014-0067)

2014-03-03 Thread Tom Lane
Noah Misch writes: > On Mon, Mar 03, 2014 at 01:29:00AM -0500, Tom Lane wrote: >> What I was envisioning was that we'd be relying on the permissions of the >> containing directory to keep out bad guys. Permissions on the socket >> itself might be sufficient, but what does it save us to assume tha

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Andres Freund writes: > On 2014-03-03 19:15:27 -0500, Tom Lane wrote: >> This greatly >> ameliorates the snapshot-skew problems that arise from its habit of doing >> some things for itself and other things via backend-internal functions >> (which historically used SnapshotNow and now use a fresh M

Re: [HACKERS] Row-security on updatable s.b. views

2014-03-03 Thread Craig Ringer
On 02/25/2014 01:28 AM, Dean Rasheed wrote: > On 13 February 2014 04:12, Craig Ringer wrote: >> >> It's crashing while pulling up the query over "emp" (hl7.employee) and >> "part" (hl7.participation). >> >> Given the simplicity of what the row-security code its self is doing, >> I'm wondering if t

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Andres Freund
On 2014-03-03 20:32:13 -0500, Tom Lane wrote: > > Afair (I really haven't rechecked) all the actions that have a changed > > locklevels affect things that pg_dump recreates clientside, using a > > repeatable read snapshot, so there shouldn't be much change there? > > You're missing the point entir

Re: [HACKERS] Securing "make check" (CVE-2014-0067)

2014-03-03 Thread Tom Lane
I wrote: > Placing the socket anywhere besides the default location will require > setting PGHOST anyway, so I don't see that this argument holds much water. > The cleanup aspect is likewise not that exciting; pg_regress creates a lot > of stuff it doesn't remove. There's another point here, if yo

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Peter Geoghegan
On Mon, Mar 3, 2014 at 5:09 PM, Josh Berkus wrote: > On 03/03/2014 05:07 PM, Peter Geoghegan wrote: >>> Primary value is that in theory the hstore2 opclasses are available >>> *now*, as opposed to a year from now. >> >> Well, yes, that's right. Although we cannot assume that VODKA will get >> into

Re: [HACKERS] ALTER TABLE lock strength reduction patch is unsafe

2014-03-03 Thread Tom Lane
Andres Freund writes: > On 2014-03-03 20:32:13 -0500, Tom Lane wrote: >> You're missing the point entirely if you think pg_dump recreates >> everything client-side. > No, I am not obviously not thinking that. What I mean is that the things > that actually change their locking requirement in the

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Andrew Dunstan
On 03/03/2014 07:50 PM, Peter Geoghegan wrote: On Fri, Feb 28, 2014 at 2:12 PM, Peter Geoghegan wrote: In order to make a rational decision to do the work incrementally, we need to know what we're putting off until 9.5. AFAICT, we have these operator classes that work fine with jsonb for the p

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Josh Berkus
On 03/03/2014 06:17 PM, Peter Geoghegan wrote: > Good. Hopefully you also mean that you recognize the dilemma referred > to above - that the hstore code reuse made a certain amount of sense, > and that more than likely the best way forward is to work out a way to > make it work. I'm not immediately

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Peter Geoghegan
On Mon, Mar 3, 2014 at 6:54 PM, Andrew Dunstan wrote: > My aim for 9.4, given constraints of both the development cycle and my time > budget, has been to get jsonb to a point where it has equivalent > functionality to json, so that nobody is forced to say "well I'll have to > use json because it l

Re: [HACKERS] Patch: show relation and tuple infos of a lock to acquire

2014-03-03 Thread Amit Kapila
On Mon, Mar 3, 2014 at 3:46 PM, Andres Freund wrote: > On 2014-03-01 13:29:18 +0530, Amit Kapila wrote: >> With new patch, the message while updating locked rows will be displayed >> as below: >> >> LOG: process 364 still waiting for ShareLock on transaction 678 after >> 1014.000ms >> CONTEXT: w

Re: [HACKERS] jsonb and nested hstore

2014-03-03 Thread Peter Geoghegan
On Mon, Mar 3, 2014 at 7:39 PM, Peter Geoghegan wrote: >> But that's really just a start. Frankly, I think we need to >> think a lot harder about how we want to be able to index this sort of data. >> The proposed hstore operators appear to me to be at best just scratching the >> surface of that. I

Re: [HACKERS] GSoC proposal - "make an unlogged table logged"

2014-03-03 Thread Fabrízio de Royes Mello
On Mon, Mar 3, 2014 at 2:40 PM, Hannu Krosing wrote: > > On 03/03/2014 05:22 PM, Tom Lane wrote: > > Stephen Frost writes: > ... > >> ISTR the discussion going something along the lines of "we'd have to WAL > >> log the entire table to do that, and if we have to do that, what's the > >> point?".

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2014-03-03 Thread Ashutosh Bapat
On Mon, Mar 3, 2014 at 9:13 PM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > On Sat, Mar 1, 2014 at 9:04 PM, Stephen Frost > wrote: > > > Erm, my thought was to use a select() loop which sends out I/O requests > > > and then loops around waiting to see who finishes it.

  1   2   >