Re: [HACKERS] Background writer process

2003-11-19 Thread Zeugswetter Andreas SB SD
> >>1. Open WAL files with O_SYNC|O_DIRECT or O_SYNC(Not sure if > > Without grouping WAL writes that does not fly. Iff however such grouping > > is implemented that should deliver optimal performance. I don't think flushing > > WAL to the OS early (before a tx commits) is necessary, since writi

Re: [HACKERS] Background writer process

2003-11-18 Thread Shridhar Daithankar
Zeugswetter Andreas SB SD wrote: 1. Open WAL files with O_SYNC|O_DIRECT or O_SYNC(Not sure if Without grouping WAL writes that does not fly. Iff however such grouping is implemented that should deliver optimal performance. I don't think flushing WAL to the OS early (before a tx commits) is necess

Re: [HACKERS] Background writer process

2003-11-18 Thread Zeugswetter Andreas SB SD
> If the background writer uses fsync, it can write and allow the buffer > to be reused and fsync later, while if we use O_SYNC, we have to wait > for the O_SYNC write to happen before reusing the buffer; > that will be slower. You can forget O_SYNC for datafiles for now. There would simply be

Re: [HACKERS] Background writer process

2003-11-18 Thread Shridhar Daithankar
Bruce Momjian wrote: Shridhar Daithankar wrote: On Friday 14 November 2003 22:10, Jan Wieck wrote: Shridhar Daithankar wrote: On Friday 14 November 2003 03:05, Jan Wieck wrote: For sure the sync() needs to be replaced by the discussed fsync() of recently written files. And I think the algorith

Re: [HACKERS] Background writer process

2003-11-17 Thread Bruce Momjian
Shridhar Daithankar wrote: > On Friday 14 November 2003 22:10, Jan Wieck wrote: > > Shridhar Daithankar wrote: > > > On Friday 14 November 2003 03:05, Jan Wieck wrote: > > >> For sure the sync() needs to be replaced by the discussed fsync() of > > >> recently written files. And I think the algorith

Re: [HACKERS] Background writer process

2003-11-17 Thread Zeugswetter Andreas SB SD
> 1. Open WAL files with O_SYNC|O_DIRECT or O_SYNC(Not sure if Without grouping WAL writes that does not fly. Iff however such grouping is implemented that should deliver optimal performance. I don't think flushing WAL to the OS early (before a tx commits) is necessary, since writing 8k or 256

Re: [HACKERS] Background writer process

2003-11-17 Thread Shridhar Daithankar
On Friday 14 November 2003 22:10, Jan Wieck wrote: > Shridhar Daithankar wrote: > > On Friday 14 November 2003 03:05, Jan Wieck wrote: > >> For sure the sync() needs to be replaced by the discussed fsync() of > >> recently written files. And I think the algorithm how much and how often > >> to flus

Re: [HACKERS] Background writer process

2003-11-14 Thread Bruce Momjian
Tom Lane wrote: > Bruce Momjian <[EMAIL PROTECTED]> writes: > > Shridhar Daithankar wrote: > >> Having fsync for regular data files and sync for WAL segment a comfortable > >> compramise? Or this is going to use fsync for all of them. > > > I think we still need sync() for WAL because sometimes

Re: [HACKERS] Background writer process

2003-11-14 Thread Tom Lane
Bruce Momjian <[EMAIL PROTECTED]> writes: > Shridhar Daithankar wrote: >> Having fsync for regular data files and sync for WAL segment a comfortable >> compramise? Or this is going to use fsync for all of them. > I think we still need sync() for WAL because sometimes backends are > going to have

Re: [HACKERS] Background writer process

2003-11-14 Thread Bruce Momjian
Shridhar Daithankar wrote: > On Friday 14 November 2003 03:05, Jan Wieck wrote: > > For sure the sync() needs to be replaced by the discussed fsync() of > > recently written files. And I think the algorithm how much and how often > > to flush can be significantly improved. But after all, this does

Re: [HACKERS] Background writer process

2003-11-14 Thread Jan Wieck
Shridhar Daithankar wrote: On Friday 14 November 2003 03:05, Jan Wieck wrote: For sure the sync() needs to be replaced by the discussed fsync() of recently written files. And I think the algorithm how much and how often to flush can be significantly improved. But after all, this does not change th

Re: [HACKERS] Background writer process

2003-11-13 Thread Shridhar Daithankar
On Friday 14 November 2003 03:05, Jan Wieck wrote: > For sure the sync() needs to be replaced by the discussed fsync() of > recently written files. And I think the algorithm how much and how often > to flush can be significantly improved. But after all, this does not > change the real checkpointing

Re: [HACKERS] Background writer process

2003-11-13 Thread Bruce Momjian
Jan Wieck wrote: > That is all right and as said, how often, how much and how forced we do > the IO can all be configurable and as flexible as people see fit. But > whether you use sync(), fsync(), fdatasync(), O_SYNC, O_DSYNC or > posix_fadvise(), somewhere you have to do the write(). And that

Re: [HACKERS] Background writer process

2003-11-13 Thread Bruce Momjian
Kurt Roeckx wrote: > On Thu, Nov 13, 2003 at 05:39:32PM -0500, Bruce Momjian wrote: > > Jan Wieck wrote: > > > Bruce Momjian wrote: > > > > He found that write() itself didn't encourage the kernel to write the > > > > buffers to disk fast enough. I think the final solution will be to use > > > > f

Re: [HACKERS] Background writer process

2003-11-13 Thread Jan Wieck
Kurt Roeckx wrote: On Thu, Nov 13, 2003 at 05:39:32PM -0500, Bruce Momjian wrote: Jan Wieck wrote: > Bruce Momjian wrote: > > He found that write() itself didn't encourage the kernel to write the > > buffers to disk fast enough. I think the final solution will be to use > > fsync or O_SYNC. > > w

Re: [HACKERS] Background writer process

2003-11-13 Thread Kurt Roeckx
On Thu, Nov 13, 2003 at 05:39:32PM -0500, Bruce Momjian wrote: > Jan Wieck wrote: > > Bruce Momjian wrote: > > > He found that write() itself didn't encourage the kernel to write the > > > buffers to disk fast enough. I think the final solution will be to use > > > fsync or O_SYNC. > > > > write(

Re: [HACKERS] Background writer process

2003-11-13 Thread Bruce Momjian
Jan Wieck wrote: > Bruce Momjian wrote: > > > Kurt Roeckx wrote: > >> On Thu, Nov 13, 2003 at 04:35:31PM -0500, Jan Wieck wrote: > >> > For sure the sync() needs to be replaced by the discussed fsync() of > >> > recently written files. And I think the algorithm how much and how often > >> > to f

Re: [HACKERS] Background writer process

2003-11-13 Thread Jan Wieck
Bruce Momjian wrote: Kurt Roeckx wrote: On Thu, Nov 13, 2003 at 04:35:31PM -0500, Jan Wieck wrote: > For sure the sync() needs to be replaced by the discussed fsync() of > recently written files. And I think the algorithm how much and how often > to flush can be significantly improved. But after

Re: [HACKERS] Background writer process

2003-11-13 Thread Bruce Momjian
Kurt Roeckx wrote: > On Thu, Nov 13, 2003 at 04:35:31PM -0500, Jan Wieck wrote: > > For sure the sync() needs to be replaced by the discussed fsync() of > > recently written files. And I think the algorithm how much and how often > > to flush can be significantly improved. But after all, this doe

Re: [HACKERS] Background writer process

2003-11-13 Thread Kurt Roeckx
On Thu, Nov 13, 2003 at 04:35:31PM -0500, Jan Wieck wrote: > For sure the sync() needs to be replaced by the discussed fsync() of > recently written files. And I think the algorithm how much and how often > to flush can be significantly improved. But after all, this does not > change the real ch