On Fri, 29 Sep 2000, Andrea Arcangeli wrote:
> On Fri, Sep 29, 2000 at 11:39:18AM -0300, Rik van Riel wrote:
> > OK, good to see that we agree on the fact that we
> > should age and swapout all pages equally agressively.
>
> Actually I think we should start looking at the mapped stuff
> _only_ wh
On Fri, Sep 29, 2000 at 11:39:18AM -0300, Rik van Riel wrote:
> OK, good to see that we agree on the fact that we
> should age and swapout all pages equally agressively.
Actually I think we should start looking at the mapped stuff _only_ when the
I/O cache aging is relevant. If the I/O cache agin
On Thu, 28 Sep 2000, Andrea Arcangeli wrote:
> On Thu, Sep 28, 2000 at 08:16:32AM -0300, Rik van Riel wrote:
> > Andrea, I have the strong impression that your idea of
> > memory balancing is based on the idea that the OS should
> > out-smart the application instead of looking at the usage
> > pat
> "ingo" == Ingo Molnar <[EMAIL PROTECTED]> writes:
Hi
ingo> 2) introducing sys_flush(), which flushes pages from the pagecache.
It is not supposed that mincore can do that (yes, just now it is not
implemented, but the interface is there to do that)?
Just curious.
--
In theory, practice
On Thu, Sep 28, 2000 at 05:13:59PM +0200, Ingo Molnar wrote:
> Can anyone see any problems with the concept of this approach? This can be
It works only on top of a filesystem while all the checkpointing clever stuff
is done internally by the DB (infact it _needs_ O_SYNC when it works on the
fs).
On Thu, 28 Sep 2000, Andrea Arcangeli wrote:
> The DBMS uses shared SCSI disks across multiple hosts on the same SCSI
> bus and synchronize the distributed cache via TCP. Tell me how to do
> that with the OS cache and mmap.
this could be supported by:
1) mlock()-ing the whole mapping.
2) intr
On Thu, Sep 28, 2000 at 01:31:40PM +0200, Ingo Molnar wrote:
> if the shm contains raw I/O data, then thats flawed application design -
> an mmap()-ed file should be used instead. Shm is equivalent to shared
The DBMS uses shared SCSI disks across multiple hosts on the same SCSI bus
and synchroniz
On Thu, Sep 28, 2000 at 08:16:32AM -0300, Rik van Riel wrote:
> Andrea, I have the strong impression that your idea of
> memory balancing is based on the idea that the OS should
> out-smart the application instead of looking at the usage
> pattern of the pages in memory.
Not sure what you mean wi
On Thu, Sep 28, 2000 at 07:08:51AM -0300, Rik van Riel wrote:
> taking care of this itself. But this is not something the OS
> should prescribe to the application.
Agreed.
> (unless the SHM users tell you that this is the normal way
> they use SHM ... but as Christoph just told us, it isn't)
sh
On Thu, 28 Sep 2000, Rik van Riel wrote:
> The OS has no business knowing what's inside that SHM page.
exactly.
> IF the shm contains I/O cache, maybe you're right. However,
> until you know that this is the case, optimising for that
> situation just doesn't make any sense.
if the shm contain
On Thu, 28 Sep 2000, Rik van Riel wrote:
> On Wed, 27 Sep 2000, Andrea Arcangeli wrote:
> > But again: if the shm contains I/O cache it should be released
> > and not swapped out. Swapping out shmfs that contains I/O cache
> > would be exactly like swapping out page-cache.
>
> The OS has no bus
On Wed, 27 Sep 2000, Andrea Arcangeli wrote:
> On Wed, Sep 27, 2000 at 10:11:43AM +0200, Christoph Rohland wrote:
> > I just checked one oracle system and it did not lock the memory. And I
>
> If that memory is used for I/O cache then such memory should
> released when the system runs into swap i
On Wed, Sep 27, 2000 at 12:25:44PM -0600, Erik Andersen wrote:
> Or sysinfo(2). Same thing...
sysinfo structure doesn't export the number of active pages in the system.
Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
On Wed Sep 27, 2000 at 07:42:00PM +0200, Andrea Arcangeli wrote:
>
> You should of course poll the /proc/meminfo. (/proc/meminfo works in O(1) in
> 2.4.x so it's just the overhead of a read syscall)
Or sysinfo(2). Same thing...
-Erik
--
Erik B. Andersen email: [EMAIL PROTECTED]
--This mes
On Wed, Sep 27, 2000 at 06:56:42PM +0200, Christoph Rohland wrote:
> Yes, but how does the application detect that it should free the mem?
The trivial way is not to detect it and to allow the user to select how much
memory it will use as cache and to take it locked and then don't care (he will
ha
Andrea Arcangeli <[EMAIL PROTECTED]> writes:
> On Wed, Sep 27, 2000 at 10:11:43AM +0200, Christoph Rohland wrote:
> > I just checked one oracle system and it did not lock the memory. And I
>
> If that memory is used for I/O cache then such memory should
> released when the system runs into swap
On Wed, Sep 27, 2000 at 10:11:43AM +0200, Christoph Rohland wrote:
> I just checked one oracle system and it did not lock the memory. And I
If that memory is used for I/O cache then such memory should released when the
system runs into swap instead of swapping it out too (otherwise it's not cache
Ingo Molnar <[EMAIL PROTECTED]> writes:
> On 27 Sep 2000, Christoph Rohland wrote:
>
> > Nobody should rely on shm swapping for productive use. But you have
> > changing/increasing loads on application servers and out of a sudden
> > you run oom. In this case the system should behave and it is _
On 27 Sep 2000, Christoph Rohland wrote:
> Nobody should rely on shm swapping for productive use. But you have
> changing/increasing loads on application servers and out of a sudden
> you run oom. In this case the system should behave and it is _very_
> good to have a smooth behaviour.
it might
Andrea Arcangeli <[EMAIL PROTECTED]> writes:
> Said that I heard of real world programs that have a .text larger than 2G
=:-O
> I know Oracle (and most other DB) are very shm intensive. However
> the fact you say the shm is not locked in memory is really a news to
> me. I really remembered tha
On Tue, Sep 26, 2000 at 10:00:12PM +0200, Peter Osterlund wrote:
> Therefore, no matter what algorithm you use in elevator_linus() the total
> number of seeks should be the same.
It isn't. There's a big difference between the two algorithms and all your
previous emails was completly correct about
On Tue, 26 Sep 2000, Andrea Arcangeli wrote:
: > smaller with your algorithm? (I later realized that request merging is
: > done before the elevator function kicks in, so your algorithm should
:
: Not sure what you mean. There are two cases: the bh is merged, or
: the bh will be queued in a new
Andrea Arcangeli <[EMAIL PROTECTED]> writes:
> Could you tell me what's wrong in having an app with a 1.5G mapped executable
> (or a tiny executable but with a 1.5G shared/private file mapping if you
> prefer),
O.K. that sound more reasonable. I was reading image as program
text... and a 1.5GB p
On Tue, Sep 26, 2000 at 06:20:47PM +0200, Christoph Rohland wrote:
> O.K. that sound more reasonable. I was reading image as program
> text... and a 1.5GB program text is a something I never have seen (and
> hopefully will never see :-)
:)
>From the shrink_mmap complexity of the algorithm point
On Tue, Sep 26, 2000 at 12:14:18AM +0200, Andrea Arcangeli wrote:
> On Mon, Sep 25, 2000 at 10:52:08PM +0200, Peter Osterlund wrote:
> > Do you know why? Is it because the average seek distance becomes
>
> Good question. No I don't know why right now. I'll try again just to be 200%
> sure and I'l
On Tue, Sep 26, 2000 at 08:54:23AM +0200, Christoph Rohland wrote:
> "Stephen C. Tweedie" <[EMAIL PROTECTED]> writes:
>
> > Hi,
> >
> > On Mon, Sep 25, 2000 at 09:32:42PM +0200, Andrea Arcangeli wrote:
> >
> > > Having shrink_mmap that browse the mapped page cache is useless
> > > as having shr
Peter Osterlund wrote:
> Btw, does anyone know how the seek time depends on the seek distance
> on modern disk hardware?
I know very little but I've seen it before on this list.
For larger seeks, the head is accelerated then decelerated to roughly
the right track. That time is approx. the squar
"Stephen C. Tweedie" <[EMAIL PROTECTED]> writes:
> Hi,
>
> On Mon, Sep 25, 2000 at 09:32:42PM +0200, Andrea Arcangeli wrote:
>
> > Having shrink_mmap that browse the mapped page cache is useless
> > as having shrink_mmap browsing kernel memory and anonymous pages
> > as it does in 2.2.x as far
On Mon, Sep 25, 2000 at 04:18:13PM -0700, Linus Torvalds wrote:
>
>
> On Tue, 26 Sep 2000, Andrea Arcangeli wrote:
> >
> > The machine will run low on memory as soon as I read 200mbyte from disk.
>
> So?
>
> Yes, at that point we'll do the LRU dance. Then we won't be low on memory
> any more
On Tue, 26 Sep 2000, Andrea Arcangeli wrote:
>
> The machine will run low on memory as soon as I read 200mbyte from disk.
So?
Yes, at that point we'll do the LRU dance. Then we won't be low on memory
any more, and we won't do the LRU dance any more. What's the magic in
zoneinfo that makes it
On Mon, Sep 25, 2000 at 03:30:10PM -0700, Linus Torvalds wrote:
> On Tue, 26 Sep 2000, Andrea Arcangeli wrote:
> >
> > I'm talking about the fact that if you have a file mmapped in 1.5G of RAM
> > test9 will waste time rolling between LRUs 384000 pages, while classzone
> > won't ever see 1 of tho
On Tue, Sep 26, 2000 at 12:30:28AM +0200, Juan J. Quintela wrote:
> Which is completely wrong if the program uses _any not completely_
> unusual locality of reference. Think twice about that, it is more
> probable that you need more that 300MB of filesystem cache that you
> have an aplication tha
On Tue, 26 Sep 2000, Andrea Arcangeli wrote:
> On Mon, Sep 25, 2000 at 08:54:57PM +0100, Stephen C. Tweedie wrote:
> > basically the whole of memory is data cache, some of which is mapped
> > and some of which is not?
>
> As as said in the last email aging on the cache is supposed to that.
>
>
On Mon, Sep 25, 2000 at 07:26:56PM -0300, Rik van Riel wrote:
> IMHO this is a minor issue because:
I don't think it's a minor issue.
If you don't have reschedule point in your equivalent of shrink_mmap and this
1.5G will happen to be consecutive in the lru order (quite probably if it's
been pag
On Mon, Sep 25, 2000 at 08:54:57PM +0100, Stephen C. Tweedie wrote:
> OK, and here's another simple real life example. A 2GB RAM machine
> running something like Oracle with a hundred client processes all
> shm-mapping the same shared memory segment.
Oracle takes the SHM locked, and it will neve
On Tue, 26 Sep 2000, Andrea Arcangeli wrote:
>
> I'm talking about the fact that if you have a file mmapped in 1.5G of RAM
> test9 will waste time rolling between LRUs 384000 pages, while classzone
> won't ever see 1 of those pages until you run low on fs cache.
What drugs are you on? Nobody l
> "andrea" == Andrea Arcangeli <[EMAIL PROTECTED]> writes:
Hi
andrea> I'm talking about the fact that if you have a file mmapped in 1.5G of RAM
andrea> test9 will waste time rolling between LRUs 384000 pages, while classzone
andrea> won't ever see 1 of those pages until you run low on fs cac
On Tue, 26 Sep 2000, Andrea Arcangeli wrote:
> On Mon, Sep 25, 2000 at 04:26:17PM -0300, Rik van Riel wrote:
> > > > It doesn't --- that is part of the design. The vm scanner propagates
> > >
> > > And that's the inferior part of the design IMHO.
> >
> > Indeed, but physical page based aging is
On Mon, Sep 25, 2000 at 04:26:17PM -0300, Rik van Riel wrote:
> > > It doesn't --- that is part of the design. The vm scanner propagates
> >
> > And that's the inferior part of the design IMHO.
>
> Indeed, but physical page based aging is a definate
> 2.5 thing ... ;(
I'm talking about the fac
On Mon, Sep 25, 2000 at 11:28:55PM +0200, Jens Axboe wrote:
> q->plug_device_fn(q, ...);
> list_add(...)
> generic_unplug_device(q);
>
> would suffice in scsi_lib for now.
It looks sane to me.
Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a mess
On Mon, Sep 25, 2000 at 10:52:08PM +0200, Peter Osterlund wrote:
> Do you know why? Is it because the average seek distance becomes
Good question. No I don't know why right now. I'll try again just to be 200%
sure and I'll let you know the results.
> smaller with your algorithm? (I later realize
On Mon, Sep 25 2000, Andrea Arcangeli wrote:
> > The scsi layer currently "manually" does a list_add on the queue itself,
> > which doesn't look too healthy.
>
> It's grabbing the io_request_lock so it looks healthy for now :)
It's safe alright, but if we want to do the generic_unplug_queue
inst
On Sun, 24 Sep 2000, Linus Torvalds wrote:
[directories in pagecache on ext2]
> > I'll do it and post the result tomorrow. I bet that there will be issues
> > I've overlooked (stuff that happens to work on UFS, but needs to be more
> > general for ext2), so it's going as "very alpha", but hey
Andrea Arcangeli <[EMAIL PROTECTED]> writes:
> The new elevator ordering algorithm returns me much better numbers
> than the CSCAN one with tiobench.
Do you know why? Is it because the average seek distance becomes
smaller with your algorithm? (I later realized that request merging is
done befor
Hi,
On Mon, Sep 25, 2000 at 09:32:42PM +0200, Andrea Arcangeli wrote:
> Having shrink_mmap that browse the mapped page cache is useless
> as having shrink_mmap browsing kernel memory and anonymous pages
> as it does in 2.2.x as far I can tell. It's an algorithm
> complexity problem and it will w
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> On Mon, Sep 25, 2000 at 07:06:57PM +0100, Stephen C. Tweedie wrote:
> > Good. One of the problems we always had in the past, though, was that
> > getting the relative aging of cache vs. vmas was easy if you had a
> > small set of test loads, but it w
On Mon, Sep 25, 2000 at 07:06:57PM +0100, Stephen C. Tweedie wrote:
> Good. One of the problems we always had in the past, though, was that
> getting the relative aging of cache vs. vmas was easy if you had a
> small set of test loads, but it was really, really hard to find a
> balance that didn'
Hi,
On Mon, Sep 25, 2000 at 07:03:47PM +0200, Andrea Arcangeli wrote:
>
> > This really seems to be the biggest difference between the two
> > approaches right now. The FreeBSD folks believe fervently that one of
> > [ aging cache and mapped pages in the same cycle ]
>
> Right.
>
> And since
On Mon, Sep 25, 2000 at 07:21:48PM +0200, bert hubert wrote:
> Ok, sorry. Kernel development is proceding at a furious pace and I sometimes
> lose track.
No problem :).
> I seem to remember that people were impressed by classzone, but that the
> implementation was very non-trivial and hard to g
> We're talking about shrink_[id]cache_memory change. That have _nothing_ to do
> with the VM changes that happened anywhere between test8 and test9-pre6.
>
> You were talking about a different thing.
Ok, sorry. Kernel development is proceding at a furious pace and I sometimes
lose track.
> I
On Mon, Sep 25, 2000 at 07:05:02PM +0200, Ingo Molnar wrote:
> yep - and Jens i'm sorry about the outburst. Until a bug is found it's
> unrealistic to blame anything.
I think the only bug maybe to blame in the elevator is the EXCLUSIVE wakeup
thing (and I've not benchmarked it alone to see if it
On Mon, 25 Sep 2000, Linus Torvalds wrote:
> Blaming the elevator is unfair and unrealistic. [...]
yep - and Jens i'm sorry about the outburst. Until a bug is found it's
unrealistic to blame anything.
Ingo
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
th
On Mon, Sep 25, 2000 at 05:24:42PM +0100, Stephen C. Tweedie wrote:
> Your other recent complaint, that newly-swapped pages end up on the
> wrong end of the LRU lists and can't be reclaimed without cycling the
> rest of the pages in shrink_mmap, is also cured in Rik's code, by
> placing pages whic
On Mon, 25 Sep 2000, Ingo Molnar wrote:
> >
> > And a new elevator was introduced some months ago to solve this.
>
> and these are still not solved in the vanilla kernel, as recent complaints
> on l-k prove.
THE ELEVATOR IS PROBABLY NOT THE PROBLEM.
People blame the elevator for bad IO perfo
Hi,
On Mon, Sep 25, 2000 at 01:41:37AM +0200, Andrea Arcangeli wrote:
>
> Since you're talking about this I'll soon (as soon as I'll finish some other
> thing that is just work in progress) release a classzone against latest's
> 2.4.x. My approch is _quite_ different from the curren VM. Current
On Mon, Sep 25, 2000 at 06:20:40PM +0200, Ingo Molnar wrote:
> i only suggested this as a debugging helper, instead of the suggested
I don't think removing the superlock from all fs is good thing at this stage (I
agree with SCT doing it only for ext2 [that's what we mostly care about] would
be po
On Mon, 25 Sep 2000, Alexander Viro wrote:
> > i'd suggest to simply BUG() in schedule() if the superblock lock is held
> > not directly by lock_super. Holding the superblock lock is IMO quite rude
> > anyway (for performance and latency) - is there any place where we hold it
> > for a long time
On Mon, 25 Sep 2000, Ingo Molnar wrote:
>
> On Mon, 25 Sep 2000, Stephen C. Tweedie wrote:
>
> > Sorry, but in this case you have got a lot more variables than you
> > seem to think. The obvious lock is the ext2 superblock lock, but
> > there are side cases with quota and O_SYNC which are mu
On Mon, 25 Sep 2000, Stephen C. Tweedie wrote:
> Sorry, but in this case you have got a lot more variables than you
> seem to think. The obvious lock is the ext2 superblock lock, but
> there are side cases with quota and O_SYNC which are much less
> commonly triggered. That's not even starting
Hi,
On Mon, Sep 25, 2000 at 12:36:50AM +0200, bert hubert wrote:
> On Mon, Sep 25, 2000 at 12:13:42AM +0200, Andrea Arcangeli wrote:
> > On Sun, Sep 24, 2000 at 10:43:03PM +0100, Stephen C. Tweedie wrote:
> > > any form of serialisation on the quota file). This feels like rather
> > > a lot of n
On Sun, Sep 24, 2000 at 11:39:13PM -0300, Marcelo Tosatti wrote:
> - Change kmem_cache_shrink to return the number of freed pages.
I did that too extending a patch from Mark. I also removed the first_not_full
ugliness providing a LIFO behaviour to the completly freed slabs (so
kmem_cache_reap re
On Mon, Sep 25, 2000 at 04:53:05PM +0200, Ingo Molnar wrote:
> sorry - i said it was *noticed* by Dimitris. (and sent to l-k IIRC)
I didn't know.
Andrea
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
Please read the FAQ at
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> > the EXCLUSIVE thing was noticed by Dimitris i think, and it makes tons of
>
> Actually I'm the one who introduced the EXCLUSIVE thing there and I audited
sorry - i said it was *noticed* by Dimitris. (and sent to l-k IIRC)
Ingo
-
To uns
On Mon, Sep 25, 2000 at 04:29:42PM +0200, Ingo Molnar wrote:
> There is no guarantee at all that the reader will win. If reads and writes
> racing for request slots ever becomes a problem then we should introduce a
> separate read and write waitqueue.
I agree. However here I also have a in flight
On Mon, Sep 25, 2000 at 04:18:54PM +0200, Jens Axboe wrote:
> The scsi layer currently "manually" does a list_add on the queue itself,
> which doesn't look too healthy.
It's grabbing the io_request_lock so it looks healthy for now :)
Andrea
-
To unsubscribe from this list: send the line "unsubsc
On Mon, Sep 25 2000, Andrea Arcangeli wrote:
> > The sg problem was different. When sg queues a request, it invokes the
> > request_fn to handle it. But if the queue is currently plugged, the
> > scsi_request_fn will not do anything.
>
> That will explain it, yes. In the same way for correctness
On Mon, Sep 25, 2000 at 04:11:34PM +0200, Jens Axboe wrote:
> Interesting. I haven't done any serious benching with the CSCAN introduction
> in elevator_linus, I'll try that too.
Only changing that the performance decreased reproducibly from 16 to 14
mbyte/sec in the read test with 2 threads.
So
On Mon, Sep 25, 2000 at 04:08:38PM +0200, Jens Axboe wrote:
> The sg problem was different. When sg queues a request, it invokes the
> request_fn to handle it. But if the queue is currently plugged, the
> scsi_request_fn will not do anything.
That will explain it, yes. In the same way for correct
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> driver (and I very much hope that with EXCLUSIVE gone away and the
> wait_on_* fixed those hangs will go away because I don't see anything else
> wrong at this moment).
the EXCLUSIVE thing only optimizes the wakeup, it's not semantic! How
better is
On Mon, Sep 25 2000, Andrea Arcangeli wrote:
> > And a new elevator was introduced some months ago to solve this.
>
> And now that I done some benchmark it seems the major optimization consists in
> the implementation of the new _ordering_ algorithm in test2, not really from
> the removal of the
On Mon, Sep 25 2000, Andrea Arcangeli wrote:
> > i had yesterday - those were simple VM deadlocks. I dont see any deadlocks
>
> Definitely. They can't explain anything about the VM deadlocks. I was
> _only_ talking about the blkdev hangs that caused you to unplug the
> queue at each reschedule in
On Mon, Sep 25, 2000 at 03:49:52PM +0200, Jens Axboe wrote:
> And a new elevator was introduced some months ago to solve this.
And now that I done some benchmark it seems the major optimization consists in
the implementation of the new _ordering_ algorithm in test2, not really from
the removal of
On Mon, Sep 25 2000, Ingo Molnar wrote:
> > The changes made were never half-done. The recent bug fixes have
> > mainly been to remove cruft from the earlier elevator and fixing a bug
> > where the elevator insert would screw up a bit. So I'd call that fine
> > tuning or adjusting, not fixing half
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> I was _only_ talking about the blkdev hangs [...]
i guess this was just miscommunication. It never 'hung', it just performed
reads with 20k/sec or so. (without any writes being done in the
background.) A 'hang' for me is a deadlock or lockup, not a
On Mon, Sep 25, 2000 at 03:57:31PM +0200, Ingo Molnar wrote:
> i had yesterday - those were simple VM deadlocks. I dont see any deadlocks
Definitely. They can't explain anything about the VM deadlocks. I was
_only_ talking about the blkdev hangs that caused you to unplug the
queue at each resched
On Mon, 25 Sep 2000, Jens Axboe wrote:
> The changes made were never half-done. The recent bug fixes have
> mainly been to remove cruft from the earlier elevator and fixing a bug
> where the elevator insert would screw up a bit. So I'd call that fine
> tuning or adjusting, not fixing half-done s
On Mon, Sep 25 2000, Ingo Molnar wrote:
> > If you think I should delay those fixes to do something else I don't
> > agree sorry.
>
> no, i never ment it. I find it very good that those half-done changes are
The changes made were never half-done. The recent bug fixes have
mainly been to remove c
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> - sync_page(page);
> set_task_state(tsk, TASK_UNINTERRUPTIBLE);
> + sync_page(page);
> - run_task_queue(&tq_disk);
> set_task_state(tsk, TASK_UNINTERRUPTIBLE);
> + run_task
Hi,
On Mon, Sep 25, 2000 at 04:02:30AM +0200, Andrea Arcangeli wrote:
> On Sun, Sep 24, 2000 at 09:27:39PM -0400, Alexander Viro wrote:
> > So help testing the patches to them. Arrgh...
>
> I think I'd better fix the bugs that I know about before testing patches that
> tries to remove the superb
On Mon, Sep 25, 2000 at 03:10:51PM +0200, Ingo Molnar wrote:
> yep. But i dont understand why this makes any difference - the waitqueue
It makes a difference because your sleeping reads won't get the wakeup
even while they could queue their reserved read request (they have
to wait the FIFO to rol
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> > yet another elevator algorithm we need a squeaky clean VM balancer above
>
> FYI: My current tree (based on 2.4.0-test8-pre5) delivers 16mbyte/sec
> in the tiobench write test compared to clean 2.4.0-test8-pre5 that
> delivers 8mbyte/sec
great!
On Mon, Sep 25, 2000 at 12:13:08PM +0200, Ingo Molnar wrote:
>
> On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
>
> > Not sure if this is the right moment for those changes though, I'm not
> > worried about ext2 but about the other non-netoworked fses that nobody
> > uses regularly.
>
> it *is* t
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> Not sure if this is the right moment for those changes though, I'm not
> worried about ext2 but about the other non-netoworked fses that nobody
> uses regularly.
it *is* the right moment to clean these issues up. These kinds of things
are what made
On Sun, 24 Sep 2000, Linus Torvalds wrote:
> I'm not claiming that the buffer cache accesses would go away - I'm just
> saying that the unbalanced "only buffer cache" case should go away,
> because things like "find" and friends will still cause mostly page cache
> activity.
>
> (Considering
On Mon, 25 Sep 2000, Alexander Viro wrote:
>
>
> On Sun, 24 Sep 2000, Linus Torvalds wrote:
>
> > The remaining part if the directory handling. THAT is very buffer-cache
> > intensive, as the directory handling hasn't been moved over to the page
> > cache at all for ext2. Doing a large "find"
On Sun, 24 Sep 2000, Linus Torvalds wrote:
> The remaining part if the directory handling. THAT is very buffer-cache
> intensive, as the directory handling hasn't been moved over to the page
> cache at all for ext2. Doing a large "find" (or even just a "ls -l") will
> basically do purely buffer
Hmm..
Thinking some more about this issue, I actually suspect that there's a
better solution.
The fact is that GFP_BUFFER is only used for the old-fashioned buffer
block allocations, and anything that uses the page cache automatically
avoids the whole issue. As such, from a VM balancing standp
bert hubert wrote:
> On Mon, Sep 25, 2000 at 12:13:42AM +0200, Andrea Arcangeli wrote:
>
>> On Sun, Sep 24, 2000 at 10:43:03PM +0100, Stephen C. Tweedie wrote:
>>
>>> any form of serialisation on the quota file). This feels like rather
>>> a lot of new and interesting deadlocks to be introduci
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> kmem_cache_reap shrinks the slabs at _very_ low frequency. It's worthless to
> keep lots of dentries and icache into the slab internal queues until
> kmem_cache_reap kicks in again, if we free them such memory immediatly instead
> we'll run kmem_c
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> On Sun, Sep 24, 2000 at 09:27:39PM -0400, Alexander Viro wrote:
> > So help testing the patches to them. Arrgh...
>
> I think I'd better fix the bugs that I know about before testing patches that
> tries to remove the superblock_lock at this stage
On Sun, Sep 24, 2000 at 09:27:39PM -0400, Alexander Viro wrote:
> So help testing the patches to them. Arrgh...
I think I'd better fix the bugs that I know about before testing patches that
tries to remove the superblock_lock at this stage. I guess you should
re-read the email from DaveM of two d
On Sun, Sep 24, 2000 at 09:53:33PM -0300, Marcelo Tosatti wrote:
> Btw, why we need kmem_cache_shrink() inside shrink_{i,d}cache_memory ?
Because kmem_cache_free doesn't free anything. It only queues slab
objects into the partial and free part of the cachep slab queue (so that
they're ready to
On Mon, 25 Sep 2000, Andrea Arcangeli wrote:
> I'm thinking that dropping the superblock lock completly wouldn't be much more
> difficult than this mid stage. The only cases where we block in critical
> sections protected by the superblock lock is in getblk/bread (bread calls
> getblk) and ll_
On Sun, Sep 24, 2000 at 05:09:40PM -0700, Linus Torvalds wrote:
> [..] as with the
> shm_swap() thing this is probably something we do want to fix eventually.
both shm_swap and regular rw_swap_cache have the same deadlock problematic
w.r.t. __GFP_IO. We could do that on a raw device, but if we sw
On Sun, 24 Sep 2000, Linus Torvalds wrote:
> > ext2_new_block (or whatever that runs getblk with the superlock lock
> > acquired)->getblk->GFP->shrink_dcache_memory->prune_dcache->
> > prune_one_dentry->dput->dentry_iput->iput->inode->i_sb->s_op->
> > put_inode->ext2_discard_prealloc->ext2_free
On Sun, 24 Sep 2000, Linus Torvalds wrote:
>
>
> On Sun, 24 Sep 2000, Andrea Arcangeli wrote:
> >
> > On Sun, Sep 24, 2000 at 10:26:11PM +0200, Ingo Molnar wrote:
> > > where will it deadlock?
> >
> > ext2_new_block (or whatever that runs getblk with the superlock lock
> > acquired)->getblk->
On Sun, 24 Sep 2000, Andrea Arcangeli wrote:
>
> On Sun, Sep 24, 2000 at 10:26:11PM +0200, Ingo Molnar wrote:
> > where will it deadlock?
>
> ext2_new_block (or whatever that runs getblk with the superlock lock
> acquired)->getblk->GFP->shrink_dcache_memory->prune_dcache->
> prune_one_dentry->d
On Mon, Sep 25, 2000 at 12:36:50AM +0200, bert hubert wrote:
> True. But they also appear to be found and solved at an impressive rate.
We're talking about shrink_[id]cache_memory change. That have _nothing_ to do
with the VM changes that happened anywhere between test8 and test9-pre6.
You were
On Mon, Sep 25, 2000 at 12:13:42AM +0200, Andrea Arcangeli wrote:
> On Sun, Sep 24, 2000 at 10:43:03PM +0100, Stephen C. Tweedie wrote:
> > any form of serialisation on the quota file). This feels like rather
> > a lot of new and interesting deadlocks to be introducing so late in
> > 2.4. :-)
T
On Sun, Sep 24, 2000 at 10:43:03PM +0100, Stephen C. Tweedie wrote:
> any form of serialisation on the quota file). This feels like rather
> a lot of new and interesting deadlocks to be introducing so late in
> 2.4. :-)
Agreed.
Andrea
-
To unsubscribe from this list: send the line "unsubscribe
1 - 100 of 107 matches
Mail list logo
