Re: [HACKERS] PATCH: pgbench - merging transaction logs

Hi, On Sat, Mar 21, 2015 at 8:42 PM, Fabien COELHO wrote: > > Hello Didier, > >>> If fprintf takes p = 0.025 (1/40) of the time, then with 2 threads the >>> collision probability would be about 1/40 and the delayed thread would be >>> waiting for half this

Re: [HACKERS] PATCH: pgbench - merging transaction logs

n more or less the same code with the same timing after a while they will lockstep on synchronization primitives and your collision probability will be very close to 1. Moreover they will write to the same cache lines for every fprintf and this is very very bad even without atomic op

Re: [HACKERS] WALWriter active during recovery

Hi On Wed, Dec 17, 2014 at 2:39 PM, Alvaro Herrera wrote: > didier wrote: > >> On many Linux systems it may not do that much (2.6.32 and 3.2 are bad, >> 3.13 is better but still it slows the fsync). >> >> If there's a fsync in progress WALReceiver will: >>

Re: [HACKERS] WALWriter active during recovery

11:51:24.754202 close(5 20278 11:51:24.754232 <... lseek resumed> ) = 194772992 <0.349825> Yes that's a 300ms lseek... >> >> >> What other useful actions can WAL receiver do while it's waiting? It doesn't >> do much else than receive WAL, and fsyn

Re: [HACKERS] Failback to old master

> DETAIL: End of WAL reached on timeline 1 at 0/3000AE0. > > This is what I experienced in the past when I tried with promote. The old > master disconnects from the new. What am I missing? > I think you have to add recovery_target_timeline = '2' in recovery.conf with '

[HACKERS] proposal: adding a GUC for BAS_BULKREAD strategy

Hi, Currently the value is hard code to NBuffers / 4 but ISTM that with bigger shared_buffer it's too much, ie even with a DB 10 to 20 time the memory size there's a lot of tables under this limit and nightly batch reports are trashing the shared buffers cache as if there's no tomorrow. regards,

Re: [HACKERS] posix_fadvise() and pg_receivexlog

Hi > Well, I'd like to hear someone from the field complaining that > pg_receivexlog is thrashing the cache and thus reducing the performance of > some other process. Or a least a synthetic test case that demonstrates that > happening. It's not with pg_receivexlog but it's related. On a small box

Re: [HACKERS] postgresql latency & bgwriter not doing its job

hi On Thu, Sep 4, 2014 at 7:01 PM, Robert Haas wrote: > On Thu, Sep 4, 2014 at 3:09 AM, Ants Aasma wrote: >> On Thu, Sep 4, 2014 at 12:36 AM, Andres Freund >> wrote: >>> It's imo quite clearly better to keep it allocated. For one after >>> postmaster started the checkpointer successfully you d

Re: [HACKERS] OSX doesn't accept identical source/target for strcpy() anymore

ds, tom lane > > I haven't a 10.9 box for double checking but there's a gcc command line triggering the same assert for strcpy and gcc at http://lists.gnu.org/archive/html/bug-bash/2013-07/msg00011.html. Didier

[HACKERS] trivial one-off memory leak in guc-file.l ParseConfigFile

Hi fix a small memory leak in guc-file.l ParseConfigFile AbsoluteConfigLocation() return a strdup string but it's never free or referenced outside ParseConfigFile Courtesy Valgrind and Noah Misch MEMPOOL work. Regards Didier memory_leak_in_parse_config_file.patch Description: Binary

Re: [HACKERS] Freezing without write I/O

Hi, IMO it's a bug if S_UNLOCK is a not a compiler barrier. Moreover for volatile remember: https://www.securecoding.cert.org/confluence/display/seccode/DCL17-C.+Beware+of+miscompiled+volatile-qualified+variables Who is double checking compiler output? :) regards Didier On Fri, Sep 20,

Re: [HACKERS] Freezing without write I/O

Hi On Fri, Sep 20, 2013 at 5:11 PM, Andres Freund wrote: > On 2013-09-20 16:47:24 +0200, Andres Freund wrote: > > I think we should go through the various implementations and make sure > > they are actual compiler barriers and then change the documented policy. > > From a quick look > * S_UNLOCK

Re: [HACKERS] Properly initialize negative/empty cache entries in relfilenodemap

Hi, On Thu, Aug 29, 2013 at 2:35 PM, MauMau wrote: > > Great! Could anybody find the root cause for the following memory leak > problem, and if possible, fix this? > > http://www.postgresql.org/**message-id/**214653D8DF574BFEAA6ED53E545E99** > E4@maumau

Re: [HACKERS] Design proposal: fsync absorb linear slider

Hi, On Fri, Jul 26, 2013 at 3:41 PM, Greg Smith (needrant.com > wrote: > On 7/26/13 9:14 AM, didier wrote: > >> During recovery you have to load the log in cache first before applying >> WAL. >> > > Checkpoints exist to bound recovery time after a crash. Th

Re: [HACKERS] Design proposal: fsync absorb linear slider

Hi, On Fri, Jul 26, 2013 at 11:42 AM, Greg Smith wrote: > On 7/25/13 6:02 PM, didier wrote: > >> It was surely already discussed but why isn't postresql writing >> sequentially its cache in a temporary file? >> > > If you do that, reads of the data will hav

Re: [HACKERS] Design proposal: fsync absorb linear slider

NEEDED buffers keeping them dirty and so on) Didier

Re: [HACKERS] Design proposal: fsync absorb linear slider

as surely already discussed but why isn't postresql writing sequentially its cache in a temporary file? With storage random speed at least five to ten time slower it could help a lot. Thanks Didier

[HACKERS] small typo in src/backend/access/transam/xlog.c

4) tv.tv_sec) << 32; sysidentifier |= (uint32) (tv.tv_sec | tv.tv_usec); should be sysidentifier |= (uint32) (tv.tv_sec ^ tv.tv_usec); Regards Didier

Re: [HACKERS] Improvement of checkpoint IO scheduler for stable transaction responses

Hi, On Sat, Jul 20, 2013 at 6:28 PM, Greg Smith wrote: > On 7/20/13 4:48 AM, didier wrote: > >> >> That is the theory. In practice write caches are so large now, there is > almost no pressure forcing writes to happen until the fsync calls show up. > It's

Re: [HACKERS] Improvement of checkpoint IO scheduler for stable transaction responses

On Sat, Jul 20, 2013 at 6:28 PM, Greg Smith wrote: > On 7/20/13 4:48 AM, didier wrote: > >> With your tests did you try to write the hot buffers first? ie buffers >> with a high refcount, either by sorting them on refcount or at least >> sweeping the buffer list in rever

Re: [HACKERS] Improvement of checkpoint IO scheduler for stable transaction responses

buffers first it may have less work to do at fsync time, hopefully they have been written by the OS background task during the spread and are not re-dirtied by other backends. Didier

Re: [HACKERS] Foreign key wierdness

Dave Page wrote: From what Tom has said in his reponse, I think the answer for you Didier is to remap your integer columns to int8 instead of int4 and see what happens. When I get a couple of minutes I will look at putting a Serials as... Option in the type map. Thanks Dave, for all of your

Re: [HACKERS] Foreign key wierdness

it's the only thing that the 2 versions of pgAdmin are doing differently. Even the PostgreSQL logs agree with that. I'm relying on Didier for test results though as I don't have a test system I can use for this at the moment. But it gives us something to try - Didier can you c

Re: [HACKERS] Foreign key wierdness

ed to look at whether the FK checking queries need to include explicit casts ... Well, I reproduced the slowdown with some 20 to 30 different tables. Anyway, glad I could be of some help, albeit only by testing some (probably quite meaningless) border cases ... :) Regards, Didier -- Didier