Re: [HACKERS] postgresql latency & bgwriter not doing its job

Hello Aidan, If all you want is to avoid the write storms when fsyncs start happening on slow storage, can you not just adjust the kernel vm.dirty* tunables to start making the kernel write out dirty buffers much sooner instead of letting them accumulate until fsyncs force them out all at once?

Re: [HACKERS] Is this code safe?

Pavan Deolasee writes: > Can some kind of compiler optimisation reorder things such that the "else > if" expression is evaluated using the old, uninitialised value of optval? Any such behavior is patently illegal per the C standard. Not that that's always stopped compiler writers; but I think yo

[HACKERS] Is this code safe?

Someone reported an issue on XL mailing list about the following code in fe-connect.c failing on Power PC platform: 1844 if (getsockopt(conn->sock, SOL_SOCKET, SO_ERROR, 1845(char *) &optval, &optlen) == -1) 1846 { 1847

Re: [HACKERS] Dense index?

On Wed, Aug 27, 2014 at 10:16 PM, Tatsuo Ishii wrote: > I know that the alignment is required for faster memory access, but > sometimes we may want to save disk space for index to save I/O because > these days customers want to handle huge number of rows. To make index > more "dense", can we add a

Re: [HACKERS] Dense index?

> Only if you want it to crash hard on most non-Intel architectures. Of course some CPU architecture prohibits none word boundary access and we need to do either: 1) do not allow to use the option on such an architecture 2) work hard to use temporary buffer which is properly aligned Best regard

Re: [HACKERS] Dense index?

Tatsuo Ishii writes: > I know that the alignment is required for faster memory access, but > sometimes we may want to save disk space for index to save I/O because > these days customers want to handle huge number of rows. To make index > more "dense", can we add an option something like "dense in

[HACKERS] Dense index?

While looking into pg_filedump output of int4 btree index, It strikes me that in leaf pages about 25% of page is wasted because of 8 byte alignment (MAXALIGN on 64bit architecture): an index tuple consists of 8 byte tuple header + 4 byte key + 4 byte alignment, thus 4/(8+4+4) = 25% is waste. I kno

Re: [HACKERS] Missing plpgsql.o Symbols on OS X

Please add -arch x86_64 to your LD_FLAGS and CFLAGS in your make file. -- Thanks & Regards, Ashesh Vashi EnterpriseDB INDIA: Enterprise PostgreSQL Company *http://www.linkedin.com/in/asheshvashi* On Wed, Aug 27, 2014 at

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

On Thu, Aug 28, 2014 at 7:37 AM, Julien Rouhaud wrote: > > Attached v2 patch implements this approach. All the work is still done > in pg_stat_get_archiver, as I don't think that having a specific > function for that information would be really interesting. Please be sure to add that to the next

Re: [HACKERS] Specifying the unit in storage parameter

On Wed, Aug 27, 2014 at 10:59 PM, Alvaro Herrera wrote: > Not necessarily, because it's harmless. It's there for purely > aesthetical reasons, so it's your choice whether to add it or not. > Having it there is slightly easier on somebody reading the code, > perhaps. On my side, that's up to yo

Re: [HACKERS] Btree internal node data?

On Wed, Aug 27, 2014 at 7:08 PM, Tatsuo Ishii wrote: > While looking into a btree internal page using pg_filedump against an > int4 index generated pgbench, I noticed that only item 2 has length 8, > which indicates that the index tuple has only tuple header and has no > index data. In my understa

[HACKERS] Btree internal node data?

While looking into a btree internal page using pg_filedump against an int4 index generated pgbench, I noticed that only item 2 has length 8, which indicates that the index tuple has only tuple header and has no index data. In my understanding this indicates that the item is used to represent a down

[HACKERS] Re: [BUGS] Re: BUG #9555: pg_dump for tables with inheritance recreates the table with the wrong order of columns

On Wed, Aug 27, 2014 at 11:24:53AM -0400, Tom Lane wrote: > On Wed, Aug 27, 2014 at 10:40:53AM -0400, Bruce Momjian wrote: > > I looked at this issue from March and I think we need to do something. > > In summary, the problem is that tables using inheritance can be dumped > > and reloaded with col

Re: [HACKERS] re-reading SSL certificates during server reload

On Wed, Aug 27, 2014 at 6:40 AM, Magnus Hagander wrote: > On Wed, Aug 27, 2014 at 11:56 AM, Alexey Klyukin wrote: >> Greetings, >> >> Is there a strong reason to disallow reloading server key and cert files >> during the PostgreSQL reload? > > Key and cert files are loaded in the postmaster. We'd

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

Thomas Munro wrote: > Thanks, I hadn't seen this, I should have checked the archives better. > I have actually already updated my patch to handle EvalPlanQualFetch > with NOWAIT and SKIP LOCKED with isolation specs, see attached. I > will compare with Craig's and see if I screwed anything up... o

Re: [HACKERS] FOR [SHARE|UPDATE] NOWAIT may still block in EvalPlanQualFetch

Craig Ringer wrote: > FOR SHARE|UPDATE NOWAIT will still block if they have to follow a ctid > chain because the call to EvalPlanQualFetch doesn't take a param for > noWait, so it doesn't know not to block if the updated row can't be locked. Applied with some further editorialization. In another

Re: [HACKERS] Missing plpgsql.o Symbols on OS X

"David E. Wheeler" writes: > Hackers, > I’m trying to build Pavel’s plpgsql_check against the 9.4 beta on OS X 10.9, > but get these errors: > make > gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith > -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute > -Wformat-security

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

Le 25/08/2014 19:00, Gilles Darold a écrit : > Le 21/08/2014 10:17, Julien Rouhaud a écrit : >> Hello, >> >> Attached patch implements the following TODO item : >> >> Track number of WAL files ready to be archived in pg_stat_archiver >> >> However, it will track the total number of any file ready t

Re: [HACKERS] possible optimization: push down aggregates

On Wed, Aug 27, 2014 at 6:46 PM, Merlin Moncure wrote: > > Yeah: I was overthinking it. My mind was on parallel processing of > the aggregate (which is not what Pavel was proposing) because that > just happens to be what I'm working on currently -- using dblink to > decompose various aggregates

Re: [HACKERS] possible optimization: push down aggregates

On Wed, Aug 27, 2014 at 3:27 PM, Tom Lane wrote: > Merlin Moncure writes: >> associative bit just makes it easier (which is important of course!). >> mean for example can be pushed down if the 'pushed down' aggregates >> return to the count to the "reaggregator" so that you can weight the >> fina

Re: [HACKERS] possible optimization: push down aggregates

On 27 Srpen 2014, 21:41, Merlin Moncure wrote: > On Wed, Aug 27, 2014 at 2:07 PM, Pavel Stehule >> >> Are there some plans to use partitioning for aggregation? > > Besides min/max, what other aggregates (mean/stddev come to mind) > would you optimize and how would you determine which ones could be

Re: [HACKERS] delta relations in AFTER triggers

On 08/28/2014 12:03 AM, Kevin Grittner wrote: Heikki Linnakangas wrote: I suggest adding a new hook to the ParseState struct, (p_rangevar_hook ?). The planner calls it whenever it sees a reference to a table, and the hook function returns back some sort of placeholder reference to the tuplestor

Re: [HACKERS] delta relations in AFTER triggers

* Kevin Grittner (kgri...@ymail.com) wrote: > Heikki Linnakangas wrote: > > In essence, make the relations work like PL/pgSQL > > variables do. If you squint a little, the new/old relation is a variable > > from the function's point of view, and a parameter from the > > planner/executor's point of

Re: [HACKERS] delta relations in AFTER triggers

Jim Nasby wrote: > Something to keep in mind is that users will definitely think about NEW/OLD as > tables. I suspect that it won't be long after release before someone asks > why they can't create an index on it. :) I'm comfortable saying "No" to that.  But it's a good point -- I'll review err

Re: [HACKERS] delta relations in AFTER triggers

Heikki Linnakangas wrote: > On 08/27/2014 02:26 AM, Kevin Grittner wrote: >> spi-tuplestore-registry allows tuplestores, with associated name >> and TupleDesc, to be registered with the current SPI connection. >> Queries planned or executed on that connection will recognize the >> name as a tuple

Re: [HACKERS] possible optimization: push down aggregates

2014-08-27 22:27 GMT+02:00 Tom Lane : > Merlin Moncure writes: > > associative bit just makes it easier (which is important of course!). > > mean for example can be pushed down if the 'pushed down' aggregates > > return to the count to the "reaggregator" so that you can weight the > > final avera

Re: [Fwd: Re: [HACKERS] proposal: new long psql parameter --on-error-stop]

Hi I chose \? xxx, because it is related to psql features. I wrote commands: \? options \? variables comments? Regards Pavel 2014-08-26 13:48 GMT+02:00 Andres Freund : > On 2014-08-26 13:44:16 +0200, Pavel Stehule wrote: > > 2014-08-26 13:30 GMT+02:00 Petr Jelinek : > > > > > On 26/08/14 1

Re: [HACKERS] possible optimization: push down aggregates

Merlin Moncure writes: > associative bit just makes it easier (which is important of course!). > mean for example can be pushed down if the 'pushed down' aggregates > return to the count to the "reaggregator" so that you can weight the > final average. that's a lot more complicated though. The r

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

Thomas Munro wrote: > On 27 August 2014 17:18, Alvaro Herrera wrote: > > Thomas Munro wrote: > >> Yes it does, thanks Alvaro and Craig. I think the attached spec > >> reproduces the problem using that trick, ie shows NOWAIT blocking, > >> presumably in EvalPlanQualFetch (though I haven't stepped

Re: [HACKERS] possible optimization: push down aggregates

On Wed, Aug 27, 2014 at 2:46 PM, Claudio Freire wrote: > On Wed, Aug 27, 2014 at 4:41 PM, Merlin Moncure wrote: >> On Wed, Aug 27, 2014 at 2:07 PM, Pavel Stehule >> wrote: >>> Hi >>> >>> one user asked about using a partitioning for faster aggregates queries. >>> >>> I found so there is not any

Re: [HACKERS] possible optimization: push down aggregates

2014-08-27 21:46 GMT+02:00 Claudio Freire : > On Wed, Aug 27, 2014 at 4:41 PM, Merlin Moncure > wrote: > > On Wed, Aug 27, 2014 at 2:07 PM, Pavel Stehule > wrote: > >> Hi > >> > >> one user asked about using a partitioning for faster aggregates queries. > >> > >> I found so there is not any opti

Re: [HACKERS] possible optimization: push down aggregates

2014-08-27 21:41 GMT+02:00 Merlin Moncure : > On Wed, Aug 27, 2014 at 2:07 PM, Pavel Stehule > wrote: > > Hi > > > > one user asked about using a partitioning for faster aggregates queries. > > > > I found so there is not any optimization. > > > > create table x1(a int, d date); > > create table

Re: [HACKERS] possible optimization: push down aggregates

On Wed, Aug 27, 2014 at 4:41 PM, Merlin Moncure wrote: > On Wed, Aug 27, 2014 at 2:07 PM, Pavel Stehule > wrote: >> Hi >> >> one user asked about using a partitioning for faster aggregates queries. >> >> I found so there is not any optimization. >> >> create table x1(a int, d date); >> create ta

Re: [HACKERS] possible optimization: push down aggregates

On Wed, Aug 27, 2014 at 2:07 PM, Pavel Stehule wrote: > Hi > > one user asked about using a partitioning for faster aggregates queries. > > I found so there is not any optimization. > > create table x1(a int, d date); > create table x_1 ( check(d = '2014-01-01'::date)) inherits(x1); > create table

[HACKERS] possible optimization: push down aggregates

Hi one user asked about using a partitioning for faster aggregates queries. I found so there is not any optimization. create table x1(a int, d date); create table x_1 ( check(d = '2014-01-01'::date)) inherits(x1); create table x_2 ( check(d = '2014-01-02'::date)) inherits(x1); create table x_3 (

Re: [HACKERS] jsonb format is pessimal for toast compression

On Wed, Aug 27, 2014 at 1:09 AM, Arthur Silva wrote: > It won't be faster by any means, but it should definitely be incorporated > if any format changes are made (like Tom already suggested). > > I think it's important we gather at least 2 more things before making any > calls: > * Josh tests w/

Re: [HACKERS] Patch to support SEMI and ANTI join removal

Jim Nasby writes: > On 8/26/14, 8:40 AM, Heikki Linnakangas wrote: >> Just so everyone is on the same page on what kind of queries this helps >> with, here are some examples from the added regression tests: >> > -- Test join removals for semi and anti joins > CREATE TEMP TABLE b (id INT NOT NULL

Re: [HACKERS] [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins

On 8/27/14, 1:38 AM, Andres Freund wrote: It occurs to me that it'd also be nice to have some >stats available on how this is performing; perhaps a dtrace probe for >whenever we overflow to the hash table, and one that shows maximum >usage for a statement? (Presumably that's not much extra code o

Re: [HACKERS] Function to know last log write timestamp

On 8/27/14, 7:33 AM, Fujii Masao wrote: On Tue, Aug 19, 2014 at 1:07 AM, Robert Haas wrote: On Fri, Aug 15, 2014 at 7:17 AM, Fujii Masao wrote: On Fri, Aug 15, 2014 at 3:40 AM, Andres Freund wrote: On 2014-08-14 14:37:22 -0400, Robert Haas wrote: On Thu, Aug 14, 2014 at 2:21 PM, Andres Fre

Re: [HACKERS] Parallel Sequence Scan doubts

On 8/24/14, 6:22 AM, Haribabu Kommi wrote: Yes, we are mainly targeting CPU-limited sequential scans, Because of this reason only I want the worker to handle the predicates also not just reading the tuples from disk. In that case, I would suggest focusing on parallel execution of conditions re

Re: [HACKERS] delta relations in AFTER triggers

On Wed, Aug 27, 2014 at 11:51:40AM -0500, Jim Nasby wrote: > On 8/27/14, 2:23 AM, Heikki Linnakangas wrote: > >Does this make sense? In essence, make the relations work like > >PL/pgSQL variables do. If you squint a little, the new/old relation > >is a variable from the function's point of view, an

Re: [HACKERS] Similar to csvlog but not really, json logs?

On 27/08/14 18:53, Andres Freund wrote: On 2014-08-26 23:04:48 -0500, Jim Nasby wrote: On 8/26/14, 8:45 PM, Michael Paquier wrote: Hi all, As mentioned here, we support multiple logging format: http://www.postgresql.org/docs/devel/static/runtime-config-logging.html Now what about a json format

Re: [HACKERS] Patch to support SEMI and ANTI join removal

On 8/26/14, 8:40 AM, Heikki Linnakangas wrote: Just so everyone is on the same page on what kind of queries this helps with, here are some examples from the added regression tests: -- Test join removals for semi and anti joins CREATE TEMP TABLE b (id INT NOT NULL PRIMARY KEY, val INT); CREATE

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On 2014-08-27 19:00:12 +0200, Fabien COELHO wrote: > > >off: > > > >$ pgbench -p 5440 -h /tmp postgres -M prepared -c 16 -j16 -T 120 -R 180 -L > >200 > >number of skipped transactions: 1345 (6.246 %) > > > >on: > > > >$ pgbench -p 5440 -h /tmp postgres -M prepared -c 16 -j16 -T 120 -R 180 -L > >

Re: [HACKERS] pgbench throttling latency limit

[...] Yeah, something like that. I don't think it would be necessary to set statement_timeout, you can inject that in your script or postgresql.conf if you want. I don't think aborting a transaction that's already started is necessary either. You could count it as LATE, but let it finish fi

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

On 27 August 2014 17:18, Alvaro Herrera wrote: > Thomas Munro wrote: >> On 25 August 2014 02:57, Alvaro Herrera wrote: >> > Thomas Munro wrote: >> >> The difficulty of course will be testing all these racy cases >> >> reproducibly... >> > >> > Does this help? >> > http://www.postgresql.org/messa

Re: [HACKERS] postgresql latency & bgwriter not doing its job

off: $ pgbench -p 5440 -h /tmp postgres -M prepared -c 16 -j16 -T 120 -R 180 -L 200 number of skipped transactions: 1345 (6.246 %) on: $ pgbench -p 5440 -h /tmp postgres -M prepared -c 16 -j16 -T 120 -R 180 -L 200 number of skipped transactions: 1 (0.005 %) That machine is far from idle ri

Re: [HACKERS] Similar to csvlog but not really, json logs?

On 2014-08-26 23:04:48 -0500, Jim Nasby wrote: > On 8/26/14, 8:45 PM, Michael Paquier wrote: > >Hi all, > > > >As mentioned here, we support multiple logging format: > >http://www.postgresql.org/docs/devel/static/runtime-config-logging.html > >Now what about a json format logging with one json obje

Re: [HACKERS] delta relations in AFTER triggers

On 8/27/14, 2:23 AM, Heikki Linnakangas wrote: Does this make sense? In essence, make the relations work like PL/pgSQL variables do. If you squint a little, the new/old relation is a variable from the function's point of view, and a parameter from the planner/executor's point of view. It's jus

Re: [HACKERS] Similar to csvlog but not really, json logs?

On 8/26/14, 8:45 PM, Michael Paquier wrote: Hi all, As mentioned here, we support multiple logging format: http://www.postgresql.org/docs/devel/static/runtime-config-logging.html Now what about a json format logging with one json object per log entry? A single json entry would need more space t

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On 2014-08-27 19:23:04 +0300, Heikki Linnakangas wrote: > On 08/27/2014 04:20 PM, Andres Freund wrote: > >On 2014-08-27 10:17:06 -0300, Claudio Freire wrote: > >>>I think a somewhat smarter version of the explicit flushes in the > >>>hack^Wpatch I posted nearby is going to more likely to be success

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On 08/27/2014 04:20 PM, Andres Freund wrote: On 2014-08-27 10:17:06 -0300, Claudio Freire wrote: I think a somewhat smarter version of the explicit flushes in the hack^Wpatch I posted nearby is going to more likely to be successful. That path is "dangerous" (as in, may not work as intended) i

Re: [HACKERS] SKIP LOCKED DATA (work in progress)

Thomas Munro wrote: > On 25 August 2014 02:57, Alvaro Herrera wrote: > > Thomas Munro wrote: > >> The difficulty of course will be testing all these racy cases > >> reproducibly... > > > > Does this help? > > http://www.postgresql.org/message-id/51fb4305.3070...@2ndquadrant.com > > The useful tri

Re: [HACKERS] Similar to csvlog but not really, json logs?

-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 Stephen Frost wrote: > To try to clarify that a bit, as it comes across as rather opaque even > on my re-reading, consider a case where you can't have the > "credit_card_number" field ever exported to an audit or log file, but > you're required

[HACKERS] Missing plpgsql.o Symbols on OS X

Hackers, I’m trying to build Pavel’s plpgsql_check against the 9.4 beta on OS X 10.9, but get these errors: make gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -I/usr/lo

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

Em 26/08/2014 09:16, "Fujii Masao" escreveu: > > On Tue, Aug 19, 2014 at 6:37 PM, Rahila Syed wrote: > > Hello, > > Thank you for comments. > > > >>Could you tell me where the patch for "single block in one run" is? > > Please find attached patch for single block compression in one run. > > Thank

Re: [HACKERS] [BUGS] Re: BUG #9555: pg_dump for tables with inheritance recreates the table with the wrong order of columns

Bruce Momjian writes: > I looked at this issue from March and I think we need to do something. > In summary, the problem is that tables using inheritance can be dumped > and reloaded with columns in a different order from the original > cluster. Yeah ... this has been a well-understood issue for

Re: [HACKERS] pgbench throttling latency limit

On 08/27/2014 06:08 PM, Fabien COELHO wrote: I've submitted this "simple" lag limit version because being able to measure quickly and simply (un)responsiveness seems like a good idea, especially given the current state of things. Ok, fair enough. I don't think doing a "latency limit" would be s

Re: [HACKERS] pgbench throttling latency limit

As for an actual "latency limit" under throttling, this is significantly more tricky and invasive to implement... ISTM that it would mean: [...] Yeah, something like that. I don't think it would be necessary to set statement_timeout, you can inject that in your script or postgresql.conf if

Re: [HACKERS] Scaling shared buffer eviction

On Tue, Aug 26, 2014 at 10:53 AM, Amit Kapila wrote: > Today, while working on updating the patch to improve locking > I found that as now we are going to have a new process, we need > a separate latch in StrategyControl to wakeup that process. > Another point is I think it will be better to prote

Re: [HACKERS] Scaling shared buffer eviction

On Tue, Aug 26, 2014 at 11:10 AM, Tom Lane wrote: > Amit Kapila writes: >> On Tue, Aug 5, 2014 at 9:21 PM, Robert Haas wrote: I think you should get rid of BufFreelistLock completely and just decide that freelist_lck will protect everything the freeNext links, plus everything in S

Re: [HACKERS] Hardening pg_upgrade

On Wed, Aug 27, 2014 at 10:13 AM, Bruce Momjian wrote: > On Wed, Aug 27, 2014 at 09:54:11AM -0400, Robert Haas wrote: >> Having said that, there are obviously advantages for our users if we >> don't get too crazy about requiring that. I've used products in the >> past where to get from version 3

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

On Tue, Aug 26, 2014 at 8:14 AM, Fujii Masao wrote: > On Tue, Aug 19, 2014 at 6:37 PM, Rahila Syed wrote: >> Hello, >> Thank you for comments. >> >>>Could you tell me where the patch for "single block in one run" is? >> Please find attached patch for single block compression in one run. > > Thank

Re: [HACKERS] postgresql latency & bgwriter not doing its job

Hello, If all you want is to avoid the write storms when fsyncs start happening on slow storage, can you not just adjust the kernel vm.dirty* tunables to start making the kernel write out dirty buffers much sooner instead of letting them accumulate until fsyncs force them out all at once? I c

[HACKERS] Re: [BUGS] Re: BUG #9555: pg_dump for tables with inheritance recreates the table with the wrong order of columns

On Mon, Mar 17, 2014 at 07:12:12PM -0400, Noah Misch wrote: > On Fri, Mar 14, 2014 at 12:33:04PM -0300, Alvaro Herrera wrote: > > Tom Lane wrote: > > > Alvaro Herrera writes: > > > > I wonder if the real fix here is to have ALTER / INHERIT error out of > > > > the columns in B are not a prefix of

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On 2014-08-27 10:32:19 -0400, Aidan Van Dyk wrote: > On Wed, Aug 27, 2014 at 3:32 AM, Fabien COELHO wrote: > > > > > Hello Andres, > > > > [...] > >> > >> I think you're misunderstanding how spread checkpoints work. > >> > > > > Yep, definitely:-) On the other hand I though I was seeking somethi

Re: [HACKERS] pgbench throttling latency limit

On 08/27/2014 02:37 PM, Fabien COELHO wrote: As for an actual "latency limit" under throttling, this is significantly more tricky and invasive to implement... ISTM that it would mean: - if the tx is not stated an the latency is already consummed, SKIP++. - if the tx is after its schedule

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On Wed, Aug 27, 2014 at 3:32 AM, Fabien COELHO wrote: > > Hello Andres, > > [...] >> >> I think you're misunderstanding how spread checkpoints work. >> > > Yep, definitely:-) On the other hand I though I was seeking something > "simple", namely correct latency under small load, that I would expe

Re: [HACKERS] Hardening pg_upgrade

On Wed, Aug 27, 2014 at 09:54:11AM -0400, Robert Haas wrote: > Having said that, there are obviously advantages for our users if we > don't get too crazy about requiring that. I've used products in the > past where to get from version 3 to version 11 you have to upgrade > from 3 to 5, then 5 to 7,

Re: [HACKERS] [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins

Andres Freund wrote: > On 2014-08-26 22:19:47 -0400, Tom Lane wrote: > > Andres Freund writes: > > I would say that the issue most deserving of performance testing is your > > sizing of the linear-search array --- it's not obvious that 8 is a good > > size. > > It's about the size of a cacheline

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

I wrote: > Robert Haas writes: >> Hmm, I'm worried that may be an API contract violation. > Indeed it is. You could get away with it if you check the > EXEC_FLAG_EXPLAIN_ONLY flag before doing anything with visible > side-effects, but it's still pretty ugly. Actually, there's another problem th

Re: [HACKERS] replication commands and log_statements

On Tue, Aug 26, 2014 at 7:17 AM, Fujii Masao wrote: > On Wed, Aug 20, 2014 at 1:14 PM, Michael Paquier > wrote: >> On Wed, Aug 20, 2014 at 2:06 AM, Robert Haas wrote: >>> >>> On Sat, Aug 16, 2014 at 10:27 AM, Amit Kapila >>> wrote: >>> > I think ideally it would have been better if we could hav

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

Robert Haas writes: > On Mon, Aug 25, 2014 at 8:58 AM, Albe Laurenz wrote: >> Reading the code, I noticed that the pushed down UPDATE or DELETE statement >> is executed >> during postgresBeginForeignScan rather than during >> postgresIterateForeignScan. >> It probably does not matter, but is th

Re: [HACKERS] Specifying the unit in storage parameter

Fujii Masao wrote: > On Tue, Aug 26, 2014 at 3:27 AM, Alvaro Herrera > wrote: > > Fujii Masao wrote: > >> On Thu, Aug 21, 2014 at 4:20 PM, Michael Paquier > >> wrote: > > > >> > Looking at the patch, the parameter "fillfactor" in the category > >> > RELOPT_KIND_HEAP (the first element in intRelOp

Re: [HACKERS] Optimization for updating foreign tables in Postgres FDW

On Mon, Aug 25, 2014 at 8:58 AM, Albe Laurenz wrote: > Reading the code, I noticed that the pushed down UPDATE or DELETE statement > is executed > during postgresBeginForeignScan rather than during postgresIterateForeignScan. > It probably does not matter, but is there a reason to do it different

Re: [HACKERS] Hardening pg_upgrade

On Mon, Aug 25, 2014 at 3:04 PM, Tom Lane wrote: > Bruce Momjian writes: >> OK, I will move in the direction of removing 8.3 support and use a >> single query to pull schema information. I was hesistant to remove 8.3 >> support as I know we have kept pg_dump support all the way back to 7.0, >>

Re: [HACKERS] ALTER SYSTEM RESET?

On Mon, Aug 25, 2014 at 1:34 PM, Amit Kapila wrote: > On Wed, Jul 30, 2014 at 9:11 AM, Amit Kapila > wrote: >> I have verified the patch and found that it works well for >> all scenario's. Few minor suggestions: >> >> 1. >> !values to the postgresql.auto.conf file. >> !Setting the parame

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On Wed, Aug 27, 2014 at 10:20 AM, Andres Freund wrote: > On 2014-08-27 10:17:06 -0300, Claudio Freire wrote: >> > I think a somewhat smarter version of the explicit flushes in the >> > hack^Wpatch I posted nearby is going to more likely to be successful. >> >> >> That path is "dangerous" (as in, m

Re: [HACKERS] Code bug or doc bug?

Bruce Momjian wrote > On Wed, Aug 27, 2014 at 09:05:41AM -0400, Robert Haas wrote: >> Another idea is to have a command that you can run, while connected to >> a particular database, that updates the default tablespace for that >> database without actually moving any data on disk - i.e. it sets >>

Re: [HACKERS] Set new system identifier using pg_resetxlog

On Mon, Aug 25, 2014 at 4:06 PM, Heikki Linnakangas wrote: > I didn't understand this one. But it seems like the obvious solution is to > not use the consumer's system identifier as the slot name. Or rename it > afterwards. You can't use the consumer's system identifier as the slot name, because

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On 2014-08-27 10:17:06 -0300, Claudio Freire wrote: > > I think a somewhat smarter version of the explicit flushes in the > > hack^Wpatch I posted nearby is going to more likely to be successful. > > > That path is "dangerous" (as in, may not work as intended) if the > filesystem doesn't properly

Re: [HACKERS] Code bug or doc bug?

On Wed, Aug 27, 2014 at 09:05:41AM -0400, Robert Haas wrote: > Another idea is to have a command that you can run, while connected to > a particular database, that updates the default tablespace for that > database without actually moving any data on disk - i.e. it sets > pg_database.dattablespace,

Re: [HACKERS] WIP Patch for GROUPING SETS phase 1

On Mon, Aug 25, 2014 at 1:35 AM, Andrew Gierth wrote: > If you look at the latest patch post, there's a small patch in it that > does nothing but unreserve the keywords and fix ruleutils to make > deparse/parse work. The required fix to ruleutils is an example of > violating your "four kinds of ke

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On Wed, Aug 27, 2014 at 10:15 AM, Andres Freund wrote: > On 2014-08-27 10:10:49 -0300, Claudio Freire wrote: >> On Wed, Aug 27, 2014 at 6:05 AM, Fabien COELHO wrote: >> >> [...] What's your evidence the pacing doesn't work? Afaik it's the fsync >> >> that causes the problem, not the the writes th

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On 2014-08-27 10:10:49 -0300, Claudio Freire wrote: > On Wed, Aug 27, 2014 at 6:05 AM, Fabien COELHO wrote: > >> [...] What's your evidence the pacing doesn't work? Afaik it's the fsync > >> that causes the problem, not the the writes themselves. > > > > > > Hmmm. My (poor) understanding is that f

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On Wed, Aug 27, 2014 at 10:10 AM, Claudio Freire wrote: > On Wed, Aug 27, 2014 at 6:05 AM, Fabien COELHO wrote: >>> [...] What's your evidence the pacing doesn't work? Afaik it's the fsync >>> that causes the problem, not the the writes themselves. >> >> >> Hmmm. My (poor) understanding is that f

Re: [HACKERS] postgresql latency & bgwriter not doing its job

On Wed, Aug 27, 2014 at 6:05 AM, Fabien COELHO wrote: >> [...] What's your evidence the pacing doesn't work? Afaik it's the fsync >> that causes the problem, not the the writes themselves. > > > Hmmm. My (poor) understanding is that fsync would work fine if everything > was already written beforeh

Re: [HACKERS] Code bug or doc bug?

On Sun, Aug 24, 2014 at 6:26 PM, Josh Berkus wrote: > Quoth our docs > (http://www.postgresql.org/docs/9.3/static/sql-alterdatabase.html): > > "The fourth form changes the default tablespace of the database. Only > the database owner or a superuser can do this; you must also have create > privileg

Re: [HACKERS] re-reading SSL certificates during server reload

* Magnus Hagander (mag...@hagander.net) wrote: > That's certainly an issue. Potentially bigger ones are that you cannot > replace an expired certificate or CRL without a restart. +100. I had forgotten about that issue- but it definitely sucks. :( > Some of this is going to have to be at least pa

Re: [HACKERS] Function to know last log write timestamp

On Tue, Aug 19, 2014 at 1:07 AM, Robert Haas wrote: > On Fri, Aug 15, 2014 at 7:17 AM, Fujii Masao wrote: >> On Fri, Aug 15, 2014 at 3:40 AM, Andres Freund >> wrote: >>> On 2014-08-14 14:37:22 -0400, Robert Haas wrote: On Thu, Aug 14, 2014 at 2:21 PM, Andres Freund wrote: > On

Re: [HACKERS] Parallel Sequence Scan doubts

On Thu, Aug 21, 2014 at 2:47 AM, Haribabu Kommi wrote: > Implementation of "Parallel Sequence Scan" > > Approach: > > 1."Parallel Sequence Scan" can achieved by using the background > workers doing the job of actual sequence scan including the > qualification check also. > > 2. Planner generates t

Re: [HACKERS] postgresql latency & bgwriter not doing its job

Hello Amit, I see there is some merit in your point which is to make bgwriter more useful than its current form. I could see 3 top level points to think about whether improvement in any of those can improve the current situation: a. Scanning of buffer pool to find the dirty buffers that ca

Re: [HACKERS] After switching primary server while using replication slot.

On Fri, Aug 22, 2014 at 11:29 PM, Andres Freund wrote: > Hi, > > On 2014-08-20 13:14:30 -0400, Robert Haas wrote: >> On Tue, Aug 19, 2014 at 6:25 AM, Fujii Masao wrote: >> > On Mon, Aug 18, 2014 at 11:16 PM, Sawada Masahiko >> > wrote: >> >> Hi all, >> >> After switching primary serer while usi

[HACKERS] Simplify calls of pg_class_aclcheck when multiple modes are used

Hi all, In a couple of code paths we do the following to check permissions on an object: if (pg_class_aclcheck(relid, userid, ACL_USAGE) != ACLCHECK_OK && pg_class_aclcheck(relid, userid, ACL_UPDATE) != ACLCHECK_OK) ereport(ERROR, blah); Wouldn't it be better to simplify that with a singl

Re: [HACKERS] Specifying the unit in storage parameter

On Tue, Aug 26, 2014 at 3:27 AM, Alvaro Herrera wrote: > Fujii Masao wrote: >> On Thu, Aug 21, 2014 at 4:20 PM, Michael Paquier >> wrote: > >> > Looking at the patch, the parameter "fillfactor" in the category >> > RELOPT_KIND_HEAP (the first element in intRelOpts of reloptions.c) is >> > not upd

Re: [HACKERS] Audit of logout

On Sat, Aug 23, 2014 at 3:44 PM, Amit Kapila wrote: > On Tue, Aug 5, 2014 at 8:04 PM, Fujii Masao wrote: >> >> Yep, the attached patch introduces PGC_SU_BACKEND and >> changes the contexts of log_connections and log_disconnections >> to PGC_SU_BACKEND. Review? >> Thanks for reviewing the patch!

Re: [HACKERS] pgbench throttling latency limit

Hello Heikki, [...] With a latency limit on when the query should finish, as opposed to how late it can start, it's a lot easier to give a number. For example, your requirements might state that a user must always get a response to a click on a web page in 200 ms, so you set the limit to 200

Re: [HACKERS] Missing comment block at the top of streamutil.h and receivelog.h

On Wed, Aug 27, 2014 at 2:07 PM, Michael Paquier wrote: > On Wed, Aug 27, 2014 at 12:31 PM, Fujii Masao wrote: >> On Wed, Aug 27, 2014 at 10:34 AM, Michael Paquier >> wrote: >> I think that it's better to add an include guard like >> >> #ifndef RECEIVELOG_H >> #define RECEIVELOG_H >> ... >> #end

Re: [HACKERS] Sample LDIF for pg_service.conf no longer works

On Sat, Aug 16, 2014 at 5:31 AM, Noah Misch wrote: > When using pg_service.conf with LDAP, we document[1] the following sample LDIF > for populating the LDAP server: > > version:1 > dn:cn=mydatabase,dc=mycompany,dc=com > changetype:add > objectclass:top > objectclass:groupOfUniqueNames > cn:mydata

Re: [HACKERS] re-reading SSL certificates during server reload

On Wed, Aug 27, 2014 at 11:56 AM, Alexey Klyukin wrote: > Greetings, > > Is there a strong reason to disallow reloading server key and cert files > during the PostgreSQL reload? Key and cert files are loaded in the postmaster. We'd need to change that. I'm not saying that's not a good idea, but i

  1   2   >