Re: [HACKERS] checkpointer continuous flushing

2015-08-31 Thread Fabien COELHO
Hello Amit, IBM POWER-8 24 cores, 192 hardware threads RAM = 492GB Wow! Thanks for trying the patch on such high-end hardware! About the disks: what kind of HDD (RAID? speed?)? HDD write cache? What is the OS? The FS? warmup=60 Quite short, but probably ok. scale=300 Means about 4-4

Re: [HACKERS] Potential GIN vacuum bug

2015-08-31 Thread Jeff Janes
On Sun, Aug 30, 2015 at 3:57 PM, Tom Lane wrote: > Jeff Janes writes: > > On Sun, Aug 30, 2015 at 11:11 AM, Tom Lane wrote: > >> Your earlier point about how the current design throttles insertions to > >> keep the pending list from growing without bound seems like a bigger > deal > >> to worry

Re: [HACKERS] [PROPOSAL] Table Partition

2015-08-31 Thread Amit Langote
Hello, On 2015-08-30 PM 10:42, My Life wrote: > > For partitioned table's scan action, and JOIN action, we implemented > a plan node named 'PartitionExpand'. the plan node can expand the > partitioned table scan node into a list of partitions according to > the filter and conditions. and it can

Re: [HACKERS] Buildfarm failure from overly noisy warning message

2015-08-31 Thread Jeff Janes
On Tue, Jul 28, 2015 at 2:38 PM, Tom Lane wrote: > Kevin Grittner writes: > > Tom Lane wrote: > >> Kevin Grittner writes: > >>> I think a LOG entry when an autovacuum process is actually canceled > >>> has value just in case it is happening on a particular table so > >>> frequently that the ta

[HACKERS] [PROPOSAL] Effective storage of duplicates in B-tree index.

2015-08-31 Thread Anastasia Lubennikova
Hi, hackers! I'm going to begin work on effective storage of duplicate keys in B-tree index. The main idea is to implement posting lists and posting trees for B-tree index pages as it's already done for GIN. In a nutshell, effective storing of duplicates in GIN is organised as follows. Index

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Oleg Bartunov
On Mon, Aug 31, 2015 at 5:48 AM, Bruce Momjian wrote: > On Sun, Aug 30, 2015 at 10:08:06PM -0400, Bruce Momjian wrote: > > On Mon, Aug 31, 2015 at 09:53:57AM +0900, Michael Paquier wrote: > > > Well, I have had many such discussions with XC/XL folks, and that > was my > > > opinion. I ha

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-31 Thread Shulgin, Oleksandr
> > Do you still have the code somewhere around? Did it see production use? >>> >> I sent it to mailing list year ago > > > http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com > Ah, thanks! Somehow I've missed this mail. You didn't add the patc

Re: [HACKERS] [patch] Proposal for \rotate in psql

2015-08-31 Thread Daniel Verite
I wrote: > What I like in that representation is that it looks good enough > to be pasted directly into a document in a word processor. And ironically, the nice unicode borders came out all garbled in the mail, thanks to a glitch in my setup that mis-reformatted them before sending. Sorry abou

Re: [HACKERS] WIP: Rework access method interface

2015-08-31 Thread Petr Jelinek
On 2015-08-27 15:15, Alexander Korotkov wrote: On Wed, Aug 26, 2015 at 7:20 PM, Alexander Korotkov mailto:a.korot...@postgrespro.ru>> wrote: On Wed, Aug 26, 2015 at 6:50 PM, Tom Lane mailto:t...@sss.pgh.pa.us>> wrote: One thought here is that we might not want to just blindly duplic

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-31 Thread Pavel Stehule
2015-08-31 11:30 GMT+02:00 Shulgin, Oleksandr : > Do you still have the code somewhere around? Did it see production use? >>> I sent it to mailing list year ago >> >> >> http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com >> > > Ah, thanks!

Re: [HACKERS] [patch] Proposal for \rotate in psql

2015-08-31 Thread Pavel Stehule
2015-08-29 5:57 GMT+02:00 Pavel Stehule : > > > 2015-08-29 0:48 GMT+02:00 Daniel Verite : > >> Hi, >> >> This is a reboot of my previous proposal for pivoting results in psql, >> with a new patch that generalizes the idea further through a command >> now named \rotate, and some examples. >> >> So

[HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread YUriy Zhuravlev
Hello hackers Recently, we were given access to the test server is IBM, 9119-MHE with 8 CPUs * 8 cores * 8 threads. We decided to take advantage of this and to find bottlenecks for read scalability (pgbench -S). All detail you can read here: http://www.postgrespro.ru/blog/pgsql/2015/08/30/p8sc

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Andres Freund
On 2015-08-31 13:54:57 +0300, YUriy Zhuravlev wrote: > We have noticed s_lock in PinBuffer and UnpinBuffer. For the test we > rewrited PinBuffer and UnpinBuffer by atomic operations and we liked > the result. Degradation of performance almost completely disappeared, > and went scaling up to 400 cli

[HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Shulgin, Oleksandr
Hello, I often find it pity that our docs are missing any information on since when a certain GUC setting, SQL-level command or function was introduced. Clicking through the "this page in other versions" links at the top of a webpage does help, but you still need to do some guessing (binary searc

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread YUriy Zhuravlev
On Monday 31 August 2015 13:03:07 you wrote: > That's definitely not correct, you should initialize the atomics using > pg_atomic_init_u32() and write to by using pg_atomic_write_u32() - not > access them directly. This breaks the fallback paths. You right. Now it's just to silence the compiler.

Re: [HACKERS] Reducing ClogControlLock contention

2015-08-31 Thread Amit Kapila
On Wed, Aug 26, 2015 at 4:18 PM, Simon Riggs wrote: > > On 26 August 2015 at 11:40, Amit Kapila wrote: >> >> On Tue, Aug 25, 2015 at 2:21 PM, Simon Riggs wrote: >>> >>> On 22 August 2015 at 15:14, Andres Freund wrote: >> >> TransactionIdSetPageStatus() calls TransactionIdSetStatusBit(

Re: [HACKERS] Dependency between bgw_notify_pid and bgw_flags

2015-08-31 Thread Ashutosh Bapat
On Sat, Aug 8, 2015 at 7:46 PM, Robert Haas wrote: > On Wed, Aug 5, 2015 at 3:33 AM, Ashutosh Bapat > wrote: > > This idea looks good. > > Thanks. It needs testing though to see if it really works as > intended. Can you look into that? > PFA the patch containing your code changes + test modul

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Magnus Hagander
On Sun, Aug 30, 2015 at 5:35 AM, Michael Paquier wrote: > > > On Sun, Aug 30, 2015 at 5:27 AM, Bruce Momjian wrote: > >> I know I am coming in late here, but I know Heroku uses random user >> names to allow a cluster to have per-user databases without showing >> external user name details: >> [..

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Magnus Hagander
On Sat, Aug 29, 2015 at 10:27 PM, Bruce Momjian wrote: > On Tue, Jul 7, 2015 at 12:57:58PM -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2015-07-07 12:03:36 -0400, Peter Eisentraut wrote: > > >> I think the DN is analogous to the remote user name, which we don't > > >> expose for an

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Magnus Hagander
On Sun, Aug 30, 2015 at 2:53 PM, Michael Paquier wrote: > > > And the commit fest of 2015-07 is now closed with the following score: > Committed: 58. > Moved to next CF: 25. > Rejected: 9. > Returned with Feedback: 25. > Total: 117. > Thanks! > > Ugh. Good to have it closed, but it seems we're st

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Michael Paquier
On Mon, Aug 31, 2015 at 9:13 PM, Magnus Hagander wrote: > Anyway - that CF is closed, but we seem to not have *any* CF open at this > point. Should we not make 2015-11 open? Yeah, switched. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Michael Paquier
On Mon, Aug 31, 2015 at 9:04 PM, Magnus Hagander wrote: > > > On Sun, Aug 30, 2015 at 5:35 AM, Michael Paquier > wrote: >> >> >> >> On Sun, Aug 30, 2015 at 5:27 AM, Bruce Momjian wrote: >>> >>> I know I am coming in late here, but I know Heroku uses random user >>> names to allow a cluster to hav

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Magnus Hagander
On Mon, Aug 31, 2015 at 2:17 PM, Michael Paquier wrote: > On Mon, Aug 31, 2015 at 9:04 PM, Magnus Hagander > wrote: > > > > > > On Sun, Aug 30, 2015 at 5:35 AM, Michael Paquier < > michael.paqu...@gmail.com> > > wrote: > >> > >> > >> > >> On Sun, Aug 30, 2015 at 5:27 AM, Bruce Momjian wrote: > >

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Andres Freund
On 2015-08-31 21:17:48 +0900, Michael Paquier wrote: > How can you be sure as well that all such deployments would use random > CN fields and/or random usernames? We have no guarantee of that as > well. Sorry, but this is a bit ridiculous. Greetings, Andres Freund -- Sent via pgsql-hackers ma

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Andres Freund
On 2015-08-31 14:29:10 +0200, Andres Freund wrote: > On 2015-08-31 21:17:48 +0900, Michael Paquier wrote: > > How can you be sure as well that all such deployments would use random > > CN fields and/or random usernames? We have no guarantee of that as > > well. > > Sorry, but this is a bit ridicul

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Stephen Frost
* Magnus Hagander (mag...@hagander.net) wrote: > On Sat, Aug 29, 2015 at 10:27 PM, Bruce Momjian wrote: > > I can see them having problems with a user being able to see the SSL > > remote user names of all connected users. > > I'm pretty sure Heroku don't use client certificates. > > And if they

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Andres Freund
On 2015-08-31 09:06:27 -0400, Stephen Frost wrote: > Perhaps it really isn't moving the bar all that much but at least for a > number of our users, it's increasing what they have to be worrying about > ("well, we knew usernames were an issue, but now we also have to worry > about system usersnames

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Anastasia Lubennikova
31.08.2015 14:06, Shulgin, Oleksandr пишет: Hello, I often find it pity that our docs are missing any information on since when a certain GUC setting, SQL-level command or function was introduced. Clicking through the "this page in other versions" links at the top of a webpage does help, bu

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Alexander Korotkov
On Mon, Aug 31, 2015 at 3:16 PM, Michael Paquier wrote: > On Mon, Aug 31, 2015 at 9:13 PM, Magnus Hagander wrote: > > Anyway - that CF is closed, but we seem to not have *any* CF open at this > > point. Should we not make 2015-11 open? > > Yeah, switched. Is it correct to switch 2015-09 commitf

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Andres Freund
Hi, On 2015-08-31 13:06:04 +0200, Shulgin, Oleksandr wrote: > I often find it pity that our docs are missing any information on since > when a certain GUC setting, SQL-level command or function was introduced. Same here. Not sure how to display it without getting disturbing the 'flow' of the docs

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Andres Freund
On 2015-08-31 16:22:54 +0300, Alexander Korotkov wrote: > Is it correct to switch 2015-09 commitfest to inprogress now? Yea, isn't it only starting the 15th? Can we add an option to display days in the CF app? Greetings, Andres Freund -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Alexander Korotkov
On Mon, Aug 31, 2015 at 4:31 PM, Andres Freund wrote: > On 2015-08-31 16:22:54 +0300, Alexander Korotkov wrote: > > Is it correct to switch 2015-09 commitfest to inprogress now? > > Yea, isn't it only starting the 15th? AFICS, on the last developer meeting it was decided to start commitfests in

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Magnus Hagander
On Mon, Aug 31, 2015 at 3:34 PM, Alexander Korotkov < a.korot...@postgrespro.ru> wrote: > On Mon, Aug 31, 2015 at 4:31 PM, Andres Freund wrote: > >> On 2015-08-31 16:22:54 +0300, Alexander Korotkov wrote: >> > Is it correct to switch 2015-09 commitfest to inprogress now? >> >> Yea, isn't it only

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Andreas Karlsson
On 08/31/2015 03:34 PM, Alexander Korotkov wrote: +1 for making these dates more explicit Yeah, I think being explicit would make life easier for newcomers. Andreas -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgres

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Shulgin, Oleksandr
On Mon, Aug 31, 2015 at 3:17 PM, Anastasia Lubennikova < a.lubennik...@postgrespro.ru> wrote: > 31.08.2015 14:06, Shulgin, Oleksandr пишет: > > Hello, > > I often find it pity that our docs are missing any information on since > when a certain GUC setting, SQL-level command or function was introdu

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Shulgin, Oleksandr
On Mon, Aug 31, 2015 at 3:30 PM, Andres Freund wrote: > Hi, > > On 2015-08-31 13:06:04 +0200, Shulgin, Oleksandr wrote: > > I often find it pity that our docs are missing any information on since > > when a certain GUC setting, SQL-level command or function was introduced. > > Same here. Not sure

Re: [HACKERS] WIP: About CMake v2

2015-08-31 Thread YUriy Zhuravlev
Thanks all hackers. I have not heard of fundamental problems and continue its development. :) -- YUriy Zhuravlev Postgres Professional: http://www.postgrespro.com The Russian Postgres Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscr

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Tom Lane
"Shulgin, Oleksandr" writes: > I often find it pity that our docs are missing any information on since > when a certain GUC setting, SQL-level command or function was introduced. > It would be nice if we could make a script that would parse the sgml files > and for every symbol it finds it would a

Re: [HACKERS] Commitfest remaining "Needs Review" items

2015-08-31 Thread Michael Paquier
On Mon, Aug 31, 2015 at 10:31 PM, Andres Freund wrote: > On 2015-08-31 16:22:54 +0300, Alexander Korotkov wrote: >> Is it correct to switch 2015-09 commitfest to inprogress now? No, this was not correct, it is expected to begin tomorrow. > Yea, isn't it only starting the 15th? If my memory does

[HACKERS] snapshot too old, configured by time

2015-08-31 Thread Kevin Grittner
As discussed when the "proof of concept" patch was submitted during 9.5 development, here is a version intended to be considered for commit to 9.6, with the following changes: 1. It is configured using time rather than number of transactions. Not only was there unanimous agreement here that this w

[HACKERS] Better detection of staled postmaster.pid

2015-08-31 Thread Pavel Raiskup
This is most likely just a request for brainstorm. It's been reported [1] that postmaster fails to start against staled postmaster.pid after (e.g.) power outage on Fedora, its due to init system parallelism and "some" other newly started process can already have allocated the same PID as the old p

Re: [HACKERS] Better detection of staled postmaster.pid

2015-08-31 Thread Kevin Grittner
Pavel Raiskup wrote: > It's been reported [1] that postmaster fails to start against staled > postmaster.pid after (e.g.) power outage on Fedora, its due to init system > parallelism and "some" other newly started process can already have allocated > the same PID as the old postmaster had -- and

Re: [HACKERS] Buildfarm failure from overly noisy warning message

2015-08-31 Thread Tom Lane
Jeff Janes writes: > On Tue, Jul 28, 2015 at 2:38 PM, Tom Lane wrote: >> Rather than remove the "sending signal" elog entirely, I reduced it to >> DEBUG1; that will avoid log chatter for normal cases but the info can >> still be obtained at need. > This part doesn't seem right to me. Now the au

Re: [HACKERS] Better detection of staled postmaster.pid

2015-08-31 Thread David G. Johnston
On Mon, Aug 31, 2015 at 10:20 AM, Kevin Grittner wrote: > Pavel Raiskup wrote: > > > It's been reported [1] that postmaster fails to start against staled > > postmaster.pid after (e.g.) power outage on Fedora, its due to init > system > > parallelism and "some" other newly started process can al

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-31 Thread Rahila Syed
Hello, On Jul 16, 2015 1:48 AM, "Rahila Syed" wrote: > > Hello, > > Please find attached updated patch >with an interface to calculate command progress in pgstat.c. This interface currently implements VACUUM progress tracking . I have added this patch to CommitFest 2015-09. It is marked as Waiti

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Shulgin, Oleksandr
On Mon, Aug 31, 2015 at 4:01 PM, Tom Lane wrote: > "Shulgin, Oleksandr" writes: > > I often find it pity that our docs are missing any information on since > > when a certain GUC setting, SQL-level command or function was introduced. > > It would be nice if we could make a script that would pars

Re: [HACKERS] Better detection of staled postmaster.pid

2015-08-31 Thread Tom Lane
Kevin Grittner writes: > Pavel Raiskup wrote: >> It's been reported [1] that postmaster fails to start against staled >> postmaster.pid after (e.g.) power outage on Fedora, > Was the other newly started process another PostgreSQL cluster? > Was it launched under the same OS user? Yes, that's wh

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Tom Lane
"Shulgin, Oleksandr" writes: > On Mon, Aug 31, 2015 at 4:01 PM, Tom Lane wrote: >> TBH, I think this is a horrid idea. We occasionally manually add remarks >> like "since version x.y, Postgres does this". Inevitably, that just bulks >> up the documentation; and it starts to look seriously silly

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread David G. Johnston
On Mon, Aug 31, 2015 at 10:39 AM, Shulgin, Oleksandr < oleksandr.shul...@zalando.de> wrote: > On Mon, Aug 31, 2015 at 4:01 PM, Tom Lane wrote: > >> >> It'll be a real >> mess if we do that for everything. >> > > I share the fear that it could become messy, but it doesn't necessary > *have to* be

Re: [HACKERS] Adding since-version tags to the docs?

2015-08-31 Thread Andres Freund
On 2015-08-31 10:48:01 -0400, Tom Lane wrote: > The problem with this proposal is that it will add far more bloat of > the latter sort than currently-useful information; and the ratio will > get worse over time. If we add that information in sane way we should be able to remove it automatically af

Re: [HACKERS] [PROPOSAL] Effective storage of duplicates in B-tree index.

2015-08-31 Thread Tomas Vondra
Hi, On 08/31/2015 09:41 AM, Anastasia Lubennikova wrote: Hi, hackers! I'm going to begin work on effective storage of duplicate keys in B-tree index. The main idea is to implement posting lists and posting trees for B-tree index pages as it's already done for GIN. In a nutshell, effective stori

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Tomas Vondra
On 08/31/2015 12:54 PM, YUriy Zhuravlev wrote: Hello hackers Recently, we were given access to the test server is IBM, 9119-MHE with 8 CPUs * 8 cores * 8 threads. We decided to take advantage of this and to find bottlenecks for read scalability (pgbench -S). All detail you can read here: http

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Andres Freund
On 2015-08-31 17:43:08 +0200, Tomas Vondra wrote: > Well, I could test the patch on a x86 machine with 4 sockets (64 cores), but > I wonder whether it makes sense at this point, as the patch really is not > correct (judging by what Andres says). Additionally it's, for default pgbench, really mostl

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Tomas Vondra
On 08/31/2015 05:48 PM, Andres Freund wrote: On 2015-08-31 17:43:08 +0200, Tomas Vondra wrote: Well, I could test the patch on a x86 machine with 4 sockets (64 cores), but I wonder whether it makes sense at this point, as the patch really is not correct (judging by what Andres says). Additio

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Andres Freund
On 2015-08-31 17:54:17 +0200, Tomas Vondra wrote: > [scratches head] So does this mean it's worth testing the patch on x86 or > not, in it's current state? You could try if you're interested. But I don't think it's super meaningful. The patch is just a POC and rather widely incorrect. Don't get m

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread Dmitry Vasilyev
We did not got any affect on core 64 with smt = 8, and we not have a 64 -cpu x86 machine with disable HT feature. You can set scale > 1000 and with shared_buffers >> size of index pgbench_accounts_pkey. You can also increase the concurrency: not only access top of b-tree index, but also to a speci

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread YUriy Zhuravlev
On Monday 31 August 2015 17:54:17 Tomas Vondra wrote: > So does this mean it's worth testing the patch on x86 > or not, in it's current state? Its realy intersting. But you need have true 64 cores without HT. (32 core +HT not have effect) -- YUriy Zhuravlev Postgres Professional: http://www.po

Re: [HACKERS] Missing latex-longtable value

2015-08-31 Thread Bruce Momjian
On Tue, Jul 7, 2015 at 03:21:50PM -0400, Bruce Momjian wrote: > On Tue, Jul 7, 2015 at 01:05:09PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > On Tue, Jul 7, 2015 at 11:48:13AM -0400, Tom Lane wrote: > > >> It's a bug. Back-patch as needed. > > > > > Doesn't that cause translation s

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-31 Thread Qingqing Zhou
On Thu, Aug 27, 2015 at 1:01 PM, Qingqing Zhou wrote: > On Wed, Aug 26, 2015 at 5:28 PM, Tom Lane wrote: >> >> After looking at the code a bit, IMO the most reasonable thing to do is to >> include this transformation in inline_set_returning_functions(), perhaps >> renaming it to something like in

Re: [HACKERS] Our trial to TPC-DS but optimizer made unreasonable plan

2015-08-31 Thread Andres Freund
On 2015-08-19 15:14:03 -0700, Josh Berkus wrote: > Asking users to refactor their applications to add OFFSET 0 is a bit > painful, if we could take care of it via a backwards-compatibility GUC. > We have many users who are specifically using the CTE optimization > barrier to work around planner fa

Re: [HACKERS] Scaling PostgreSQL at multicore Power8

2015-08-31 Thread YUriy Zhuravlev
On Monday 31 August 2015 17:48:50 Andres Freund wrote: > Additionally it's, for default pgbench, really mostly a bottlneck after > GetSnapshotData() is fixed. You can make it a problem much earlier if > you have index nested loops over a lot of rows. 100 000 000 is a lot? Simple select query from p

Re: [HACKERS] On-demand running query plans using auto_explain and signals

2015-08-31 Thread Shulgin, Oleksandr
On Mon, Aug 31, 2015 at 12:35 PM, Pavel Stehule wrote: > >>> >>> http://www.postgresql.org/message-id/cafj8praxcs9b8abgim-zauvggqdhpzoarz5ysp1_nhv9hp8...@mail.gmail.com >>> >> >> Ah, thanks! Somehow I've missed this mail. You didn't add the patch to >> a commitfest back then I think? >> > > I h

[HACKERS] Should \o mean "everything?"

2015-08-31 Thread David Fetter
Folks, In a failed attempt to send the output of \pset to a pipe, I noticed that for reasons I find difficult to explain, not every output gets redirected with \o. At first blush, I'd consider this inconsistency as a bug. What have I missed? Cheers, David. -- David Fetter http://fetter.org/ P

[HACKERS] Is "WIN32" #defined in Cygwin builds?

2015-08-31 Thread Tom Lane
I started wondering about $subject because we are fairly schizophrenic about whether we believe this. For example, only a few lines apart in dirmod.c, there are #if defined(WIN32) || defined(__CYGWIN__) #if defined(WIN32) && !defined(__CYGWIN__) Presumably, one of these could be simplified, but

Re: [HACKERS] [PATCH] SQL function to report log message

2015-08-31 Thread dinesh kumar
Hi, On Sat, Aug 29, 2015 at 4:22 PM, Pavel Stehule wrote: > Hi > > I am starting to work review of this patch > > 2015-07-13 9:54 GMT+02:00 dinesh kumar : > >> Hi All, >> >> Greetings for the day. >> >> Would like to discuss on below feature here. >> >> Feature: >> Having an SQL function, to

Re: [HACKERS] Is "WIN32" #defined in Cygwin builds?

2015-08-31 Thread Andrew Dunstan
On 08/31/2015 02:21 PM, Tom Lane wrote: I started wondering about $subject because we are fairly schizophrenic about whether we believe this. For example, only a few lines apart in dirmod.c, there are #if defined(WIN32) || defined(__CYGWIN__) #if defined(WIN32) && !defined(__CYGWIN__) Presu

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Mason S
> > > We also a bit disappointed by Huawei position about CSN patch, we hoped > to use for our XTM. > Disappointed in what way? Moving to some sort of CSN approach seems to open things up for different future ideas. In the short term, it would mean replacing potentially large snapshots and longe

[HACKERS] Anybody have icc for IA64?

2015-08-31 Thread Tom Lane
After pushing 2c713d6e, I realized that the buildfarm isn't going to tell me anything useful about it, because the change only makes a difference for icc on ia64, and we have no such members in the buildfarm. (We've got icc, and we've got ia64, but not both in the same place.) It's fairly worriso

Re: [HACKERS] Is "WIN32" #defined in Cygwin builds?

2015-08-31 Thread Tom Lane
Andrew Dunstan writes: > On 08/31/2015 02:21 PM, Tom Lane wrote: >> I started wondering about $subject because we are fairly schizophrenic >> about whether we believe this. > No, and we've made sure not to do that ourselves, or at least I hope we > have. OK, thanks. I was wondering whether I'd

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Bruce Momjian
On Mon, Aug 31, 2015 at 02:48:31PM -0400, Mason S wrote: > I assume that future work around PG sharding probably would be more likely to > be accepted with the FDW approach. One could perhaps work on pushing down > joins, aggregates and order by, then look at any optimizations gained if code > is m

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Alexander Korotkov
On Mon, Aug 31, 2015 at 9:48 PM, Mason S wrote: > >> We also a bit disappointed by Huawei position about CSN patch, we hoped >> to use for our XTM. >> > > Disappointed in what way? Moving to some sort of CSN approach seems to > open things up for different future ideas. In the short term, it wo

Re: [HACKERS] Anybody have icc for IA64?

2015-08-31 Thread Alvaro Herrera
Tom Lane wrote: > After pushing 2c713d6e, I realized that the buildfarm isn't going to tell > me anything useful about it, because the change only makes a difference > for icc on ia64, and we have no such members in the buildfarm. (We've > got icc, and we've got ia64, but not both in the same plac

Re: [HACKERS] Information of pg_stat_ssl visible to all users

2015-08-31 Thread Peter Eisentraut
On 8/31/15 9:13 AM, Andres Freund wrote: > I'm just saying that we should strive to behave at least somewhat > consistently, and change everything at once, not piecemal. Because the > latter will not decrease the pain of migrating to a new model in a > relevant way while making the system harder to

Re: [HACKERS] Should \o mean "everything?"

2015-08-31 Thread Kevin Grittner
David Fetter wrote: > In a failed attempt to send the output of \pset to a pipe, I > noticed that for reasons I find difficult to explain, not every > output gets redirected with \o. > > At first blush, I'd consider this inconsistency as a bug. > > What have I missed? The documentation says: |

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Qingqing Zhou
On Mon, Aug 31, 2015 at 2:12 AM, Oleg Bartunov wrote: > > AFAIK, XC/XL has already some customers and that is an additional pressure > on their development team, which is now called X2. I don't exactly know how > internal Huawei's MPPDB is connected to XC/XL. > Huawei's MPPDB is based on PG-XC an

Re: [HACKERS] Should \o mean "everything?"

2015-08-31 Thread David Fetter
On Mon, Aug 31, 2015 at 07:18:02PM +, Kevin Grittner wrote: > David Fetter wrote: > > > In a failed attempt to send the output of \pset to a pipe, I > > noticed that for reasons I find difficult to explain, not every > > output gets redirected with \o. > > > > At first blush, I'd consider thi

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Josh Berkus
All, Bruce: First, let me put out there that I think the horizontal scaling project which has buy-in from the community and we're working on is infinitely better than the one we're not working on or is an underresourced fork. So we're in agreement on that. However, I think there's a lot of room f

Re: [HACKERS] [PROPOSAL] Table Partition

2015-08-31 Thread My Life
> There is already a recent proposal on hackers about partition support in > PostgreSQL by Amit Langote. > You will find the thread at > http://www.postgresql.org/message-id/55d3093c.5010...@lab.ntt.co.jp. Actually, I have seen this design before, and it was not just a design, it has been im

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Joshua D. Drake
On 08/31/2015 01:16 PM, Josh Berkus wrote: All, Bruce: I'm also going to pontificate that, for a future solution, we should not focus on write *IO*, but rather on CPU and RAM. The reason for this thinking is that, with the latest improvements in hardware and 9.5 improvements, it's increasingl

Re: [HACKERS] Should \o mean "everything?"

2015-08-31 Thread David Fetter
On Mon, Aug 31, 2015 at 07:18:02PM +, Kevin Grittner wrote: > David Fetter wrote: > > > In a failed attempt to send the output of \pset to a pipe, I > > noticed that for reasons I find difficult to explain, not every > > output gets redirected with \o. > > > > At first blush, I'd consider thi

Re: [HACKERS] security labels on databases are bad for dump & restore

2015-08-31 Thread Bruce Momjian
On Tue, Jul 28, 2015 at 04:23:36PM -0300, Alvaro Herrera wrote: > Josh Berkus wrote: > > On 07/28/2015 11:58 AM, Robert Haas wrote: > > > I'd be strongly in favour of teaching GRANT, SECURITY LABEL, COMMENT > > >> ON DATABASE, etc to recognise CURRENT_DATABASE as a keyword. Then > > >> dumping them

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Robert Haas
On Mon, Aug 31, 2015 at 4:16 PM, Josh Berkus wrote: > First, let me put out there that I think the horizontal scaling project > which has buy-in from the community and we're working on is infinitely > better than the one we're not working on or is an underresourced fork. > So we're in agreement on

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Bruce Momjian
On Tue, Jul 14, 2015 at 09:48:59AM -0700, Smitha Pamujula wrote: > This error will go away only if I install the new json_build94. > > I was under the impression that we dont need to get the json_build > libraries for 94. But

Re: [HACKERS] WIP: Access method extendability

2015-08-31 Thread Alexander Korotkov
Hackers, there is next revision of patches providing access method extendability. Now it's based on another patch which reworks access method interface. http://www.postgresql.org/message-id/capphfdsxwzmojm6dx+tjnpyk27kt4o7ri6x_4oswcbyu1rm...@mail.gmail.com Besides access method interface, major c

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Smitha Pamujula
Thank you Bruce. So far installing it before have been working well so we will continue with that plan. I think it would help if its noted somewhere in the document as it would have helped us save some time understanding why it was failing and why it was looking for json_build. On Mon, Aug 31,

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Bruce Momjian
On Mon, Aug 31, 2015 at 04:03:20PM -0700, Smitha Pamujula wrote: > Thank you Bruce. So far installing it before have been working well so we will > continue with that plan.  > > I think it would help if its noted somewhere in the document as it would have > helped us save some time understanding w

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread David E. Wheeler
On Aug 31, 2015, at 4:20 PM, Bruce Momjian wrote: >> I think it would help if its noted somewhere in the document as it would have >> helped us save some time understanding why it was failing and why it was >> looking for json_build. > > The problem is that this is a rare case where you had an

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Andrew Dunstan
On 08/31/2015 07:21 PM, David E. Wheeler wrote: On Aug 31, 2015, at 4:20 PM, Bruce Momjian wrote: I think it would help if its noted somewhere in the document as it would have helped us save some time understanding why it was failing and why it was looking for json_build. The problem is tha

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Sumedh Pathak
Hi Bruce, Sumedh from Citus Data here. > August, 2015: While speaking at SFPUG, Citus Data approached me about joining the FDW sharding team. They have been invited to the September 1 meeting, as have the XC and XL people. I'd like to add a clarification. We already tried the FDW APIs for pg_s

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Bruce Momjian
On Mon, Aug 31, 2015 at 07:28:00PM -0400, Andrew Dunstan wrote: > > > On 08/31/2015 07:21 PM, David E. Wheeler wrote: > >On Aug 31, 2015, at 4:20 PM, Bruce Momjian wrote: > > > >>>I think it would help if its noted somewhere in the document as it would > >>>have > >>>helped us save some time un

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Smitha Pamujula
pg_upgrade skipping the modules makes the most sense to me as well. On Mon, Aug 31, 2015 at 4:32 PM, Bruce Momjian wrote: > On Mon, Aug 31, 2015 at 07:28:00PM -0400, Andrew Dunstan wrote: > > > > > > On 08/31/2015 07:21 PM, David E. Wheeler wrote: > > >On Aug 31, 2015, at 4:20 PM, Bruce Momjian

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Andrew Dunstan
On 08/31/2015 07:32 PM, Bruce Momjian wrote: On Mon, Aug 31, 2015 at 07:28:00PM -0400, Andrew Dunstan wrote: On 08/31/2015 07:21 PM, David E. Wheeler wrote: On Aug 31, 2015, at 4:20 PM, Bruce Momjian wrote: I think it would help if its noted somewhere in the document as it would have help

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread Tom Lane
Andrew Dunstan writes: > On 08/31/2015 07:32 PM, Bruce Momjian wrote: >> Still, I don't know how many people are doing this, but the right fix is >> to get the names of the modules that are superceeded and tell pg_upgrade >> to skip them. > I don't think this knowledge should be hardcoded in pg_u

Re: [HACKERS] pg_upgrade + Extensions

2015-08-31 Thread David E. Wheeler
On Aug 31, 2015, at 4:58 PM, Tom Lane wrote: > In any case, there is plenty of precedent for hard-coding knowledge about > specific version updates into pg_upgrade. The question here is whether > it's feasible to handle extensions that way. I think we could reasonably > expect to know about cas

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Josh Berkus
On 08/31/2015 02:47 PM, Robert Haas wrote: > On Mon, Aug 31, 2015 at 4:16 PM, Josh Berkus wrote: >> First, let me put out there that I think the horizontal scaling project >> which has buy-in from the community and we're working on is infinitely >> better than the one we're not working on or is an

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Marc Munro
On Mon, 2015-08-31 at 22:21 +, Robert Haas wrote: > It seems to me that sharding consists of (1) breaking your data set up > into shards, (2) possibly replicating some of those shards onto > multiple machines, and then (3) being able to access the remote data > from local queries. [...] I be

Re: [HACKERS] Horizontal scalability/sharding

2015-08-31 Thread Bruce Momjian
On Mon, Aug 31, 2015 at 05:10:11PM -0700, Josh Berkus wrote: > > As far as (3) is concerned, why > > wouldn't we use the foreign data wrapper interface, and specifically > > postgres_fdw? That interface was designed for the explicit purpose of > > allowing access to remote data sources, and a lot

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-31 Thread Peter Eisentraut
On 8/20/15 9:59 AM, Andrew Dunstan wrote: > The regression tests thus passed, but should not have. It occurred to me > that if we had a test like > > select pg_config('configure') ~ '--with-libxml' as has_xml; > > in the xml tests then this failure mode would be detected. This particular cas

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-31 Thread Peter Eisentraut
On 8/24/15 9:50 AM, Tom Lane wrote: > Andrew Dunstan writes: >> On 08/23/2015 08:58 PM, Michael Paquier wrote: >>> I think that's a good thing to have, now I have concerns about making >>> this data readable for non-superusers. Cloud deployments of Postgres >>> are logically going to block the acc

Re: [HACKERS] exposing pg_controldata and pg_config as functions

2015-08-31 Thread Peter Eisentraut
On 8/25/15 11:32 PM, Joe Conway wrote: > 1.) pg_controldata() function and pg_controldata view added I don't think dumping out whatever pg_controldata happens to print as a bunch of text fields is very sophisticated. We have functionality to compute with WAL positions, for example, and they won't

  1   2   >