Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > On Mon, 2003-11-03 at 11:11, Tom Lane wrote: >> Why not? The advice says that you're going to access the data >> sequentially in the forward direction. If you're not going to back up, >> there is no point in keeping pages in cache after they've been read.

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 14:24, Manfred Spraul wrote: > Neil: what about calling posix_fadvise for the whole file immediately > after issue_xlog_fsync() in XLogWrite? According to the comment, it's > guaranteed that this will happen only once. > Or: add an posix_fadvise into issue_xlog_fsync(), for

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 09:38, Tom Lane wrote: > Neil Conway <[EMAIL PROTECTED]> writes: > > Given a Relation and an advice, this would: > > (a) propagate the advice for this relation to all the open FDs for the > > relation > > "All"? You cannot affect the FDs being used by other backends. Sorry,

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Manfred Spraul
Neil Conway wrote: The present Linux implementation doesn't do this, AFAICS -- all it does it increase the readahead for this file: AFAIK Linux uses a modified LRU that automatically puts pages that were touched only once at a lower priority than frequently accessed pages. Neil: what about ca

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 12:17, Hannu Krosing wrote: > Can you use combinations like ( FADV_NOREUS | FADV_SEQUENTIAL ) You can do an fadvise() for FADV_SEQUENTIAL, and then another fadvise() for FADV_NOREUSE. -Neil ---(end of broadcast)--- TIP 4: Do

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Hannu Krosing
Neil Conway kirjutas E, 03.11.2003 kell 18:59: > On Mon, 2003-11-03 at 11:11, Tom Lane wrote: > > Why not? The advice says that you're going to access the data > > sequentially in the forward direction. If you're not going to back up, > > there is no point in keeping pages in cache after they've

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 11:11, Tom Lane wrote: > Why not? The advice says that you're going to access the data > sequentially in the forward direction. If you're not going to back up, > there is no point in keeping pages in cache after they've been read. The advice says: "I'm going to read this da

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > On Mon, 2003-11-03 at 10:01, Tom Lane wrote: >> I would expect POSIX_FADV_SEQUENTIAL to reduce the chance that a page >> will be kept in buffer cache after it's been used. > I don't think that can be reasonably implied from the POSIX text, which > is merel

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 10:01, Tom Lane wrote: > Neil Conway <[EMAIL PROTECTED]> writes: > > POSIX_FADV_RANDOM doesn't effect the page cache, it just determines how > > aggressive the kernel is when doing readahead (at least on Linux, but > > I'd expect to see other kernels implement similar behavior

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > POSIX_FADV_RANDOM doesn't effect the page cache, it just determines how > aggressive the kernel is when doing readahead (at least on Linux, but > I'd expect to see other kernels implement similar behavior). I would expect POSIX_FADV_SEQUENTIAL to reduce th

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Tom Lane
Neil Conway <[EMAIL PROTECTED]> writes: > So what API is desirable for uses 2-4? I'm thinking of adding a new > function to the smgr API, smgradvise(). It's a little premature to be inventing APIs when you have no evidence that this will make any useful performance difference. I'd recommend a qui

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Andrew Sullivan
On Mon, Nov 03, 2003 at 08:50:00AM -0500, Neil Conway wrote: > pool or the kernel's page cache. We can implement the former easily > enough, but I don't see any feasible way to do the latter: on a high-end > machine with gigabytes of RAM but a relatively small shared_buffers > (which is the config

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Neil Conway
On Mon, 2003-11-03 at 04:21, Hannu Krosing wrote: > Neil Conway kirjutas E, 03.11.2003 kell 08:07: > > (2) ISTM that we can set POSIX_FADV_RANDOM for *all* indexes, since the > > vast majority of the accesses to them shouldn't be sequential. > > Perhaps we could do it for all _leaf_ nodes, the roo

Re: [HACKERS] adding support for posix_fadvise()

2003-11-03 Thread Hannu Krosing
Neil Conway kirjutas E, 03.11.2003 kell 08:07: > A couple days ago, Manfred Spraul mentioned the posix_fadvise() API on > -hackers: > > http://www.opengroup.org/onlinepubs/007904975/functions/posix_fadvise.html > > I'm working on making use of posix_fadvise() where appropriate. I can > think of t

Re: [HACKERS] adding support for posix_fadvise()

2003-11-02 Thread Neil Conway
On Mon, 2003-11-03 at 01:07, Neil Conway wrote: > (1) As Manfred originally noted, when we advance to a new XLOG segment, > we can use POSIX_FADV_DONTNEED to let the kernel know we won't be > accessing the old WAL segment anymore. I've attached a quick kludge of a > patch that implements this. I ha

[HACKERS] adding support for posix_fadvise()

2003-11-02 Thread Neil Conway
A couple days ago, Manfred Spraul mentioned the posix_fadvise() API on -hackers: http://www.opengroup.org/onlinepubs/007904975/functions/posix_fadvise.html I'm working on making use of posix_fadvise() where appropriate. I can think of the following places where this would be useful: (1) As Manfr