Hi.
On Wed, 2007-01-10 at 22:07 +0800, Fengguang Wu wrote:
> Thanks, Nigel.
> But I'm very sorry that the calculation in the patch was wrong.
>
> Would you give this new patch a run?
Sorry for my slowness. I just did
time find /usr/src | wc -l
again:
Without patch: 35.137, 35.104, 35.351 seco
On Wed, Jan 10, 2007 at 02:20:49PM +1100, Nigel Cunningham wrote:
> Hi.
>
> On Wed, 2007-01-10 at 09:57 +0800, Fengguang Wu wrote:
> > On Tue, Jan 09, 2007 at 08:23:32AM -0800, Linus Torvalds wrote:
> > >
> > >
> > > On Tue, 9 Jan 2007, Fengguang Wu wrote:
> > > > >
> > > > > The fastest and proba
Hi.
On Wed, 2007-01-10 at 09:57 +0800, Fengguang Wu wrote:
> On Tue, Jan 09, 2007 at 08:23:32AM -0800, Linus Torvalds wrote:
> >
> >
> > On Tue, 9 Jan 2007, Fengguang Wu wrote:
> > > >
> > > > The fastest and probably most important thing to add is some readahead
> > > > smarts to directories ---
On Tue, Jan 09, 2007 at 08:23:32AM -0800, Linus Torvalds wrote:
>
>
> On Tue, 9 Jan 2007, Fengguang Wu wrote:
> > >
> > > The fastest and probably most important thing to add is some readahead
> > > smarts to directories --- both to the htree and non-htree cases. If
> >
> > Here's is a quick hack
On Tue, 9 Jan 2007, Fengguang Wu wrote:
> >
> > The fastest and probably most important thing to add is some readahead
> > smarts to directories --- both to the htree and non-htree cases. If
>
> Here's is a quick hack to practice the directory readahead idea.
> Comments are welcome, it's a fre
On Mon, Jan 08, 2007 at 07:58:19AM -0500, Theodore Tso wrote:
> On Mon, Jan 08, 2007 at 08:35:55AM +0530, Suparna Bhattacharya wrote:
> > > Yeah, slowly-growing directories will get splattered all over the disk.
> > >
> > > Possible short-term fixes would be to just allocate up to (say) eight
> >
On Mon, Jan 08, 2007 at 05:09:34PM -0800, Paul Jackson wrote:
> Jeff wrote:
> > Something I just thought of: ATA and SCSI hard disks do their own
> > read-ahead.
>
> Probably this is wishful thinking on my part, but I would have hoped
> that most of the read-ahead they did was for stuff that happ
Jeff wrote:
> Something I just thought of: ATA and SCSI hard disks do their own
> read-ahead.
Probably this is wishful thinking on my part, but I would have hoped
that most of the read-ahead they did was for stuff that happened to be
on the cylinder they were reading anyway. So long as their rea
On Sun, 7 Jan 2007, Shawn O. Pearce wrote:
> Krzysztof Halasa <[EMAIL PROTECTED]> wrote:
> > Hmm... Perhaps it should be possible to push git updates as a pack
> > file only? I mean, the pack file would stay packed = never individual
> > files and never 256 directories?
>
> Latest Git does this.
On Mon, Jan 08, 2007 at 02:59:52PM +0100, Pavel Machek wrote:
> Hi!
>
> > > Would e2fsck -D help? What kind of optimization
> > > does it perform?
> >
> > It will help a little; e2fsck -D compresses the logical view of the
> > directory, but it doesn't optimize the physical layout on disk at all,
On Mon, Jan 08, 2007 at 07:58:19AM -0500, Theodore Tso wrote:
>
> The fastest and probably most important thing to add is some readahead
> smarts to directories --- both to the htree and non-htree cases. If
> you're using some kind of b-tree structure, such as XFS does for
> directories, prealloc
Hi!
> > Would e2fsck -D help? What kind of optimization
> > does it perform?
>
> It will help a little; e2fsck -D compresses the logical view of the
> directory, but it doesn't optimize the physical layout on disk at all,
> and of course, it won't help with the lack of readahead logic. It's
> po
On Mon, Jan 08, 2007 at 02:41:47PM +0100, Johannes Stezenbach wrote:
>
> Would e2fsck -D help? What kind of optimization
> does it perform?
It will help a little; e2fsck -D compresses the logical view of the
directory, but it doesn't optimize the physical layout on disk at all,
and of course, it
Theodore Tso wrote:
The fastest and probably most important thing to add is some readahead
smarts to directories --- both to the htree and non-htree cases. If
you're using some kind of b-tree structure, such as XFS does for
directories, preallocation doesn't help you much. Delayed allocation
ca
On Mon, Jan 08, 2007 at 08:35:55AM +0530, Suparna Bhattacharya wrote:
> > Yeah, slowly-growing directories will get splattered all over the disk.
> >
> > Possible short-term fixes would be to just allocate up to (say) eight
> > blocks when we grow a directory by one block. Or teach the
> > direct
On Sun, Jan 07, 2007 at 01:15:42AM -0800, Andrew Morton wrote:
> On Sun, 7 Jan 2007 09:55:26 +0100
> Willy Tarreau <[EMAIL PROTECTED]> wrote:
>
> > On Sat, Jan 06, 2007 at 09:39:42PM -0800, Linus Torvalds wrote:
> > >
> > >
> > > On Sat, 6 Jan 2007, H. Peter Anvin wrote:
> > > >
> > > > During ext
Robert Fitzsimons wrote:
>> Some more data on how git affects kernel.org...
>
> I have a quick question about the gitweb configuration, does the
> $projects_list config entry point to a directory or a file?
It can point to both. Usually it is either unset, and then we
do find over $projectroot,
Krzysztof Halasa <[EMAIL PROTECTED]> wrote:
> Hmm... Perhaps it should be possible to push git updates as a pack
> file only? I mean, the pack file would stay packed = never individual
> files and never 256 directories?
Latest Git does this. If the server is later than 1.4.3.3 then
the receive-pa
On Sun, 7 Jan 2007, Randy Dunlap wrote:
>
> ISTM that Linus is trying to make 2.6.20-final before LCA. We'll see.
No. Hopefully "final -rc" before LCA, but I'll do the actual 2.6.20
release afterwards. I don't want to have a merge window during LCA, as I
and many others will all be out anywa
On Sun, 7 Jan 2007, Jon Smirl wrote:
> >
> > - proper read-ahead. Right now, even if the directory is totally
> >contiguous on disk (just remove the thing that writes data to the
> >files, so that you'll have empty files instead of 8kB files), I think
> >we do those reads totally sy
On Sun, 7 Jan 2007 20:07:43 +0100 (MET) Jan Engelhardt wrote:
>
> On Jan 7 2007 10:49, Randy Dunlap wrote:
> >On Sun, 7 Jan 2007 11:50:57 +0100 (MET) Jan Engelhardt wrote:
> >> On Jan 7 2007 10:03, Willy Tarreau wrote:
> >> >On Sun, Jan 07, 2007 at 12:58:38AM -0800, H. Peter Anvin wrote:
> >> >>
On Jan 7 2007 10:49, Randy Dunlap wrote:
>On Sun, 7 Jan 2007 11:50:57 +0100 (MET) Jan Engelhardt wrote:
>> On Jan 7 2007 10:03, Willy Tarreau wrote:
>> >On Sun, Jan 07, 2007 at 12:58:38AM -0800, H. Peter Anvin wrote:
>> >> >[..]
>> >> >entries in directories with millions of files on disk. I'm not
On Sun, 7 Jan 2007, Linus Torvalds wrote:
>
> A year or two ago I did a totally half-assed code for the non-hashed
> readdir that improved performance by an order of magnitude for ext3 for a
> test-case of mine, but it was subtly buggy and didn't do the hashed case
> AT ALL.
Btw, this isn't
With my gitweb caching changes this isn't as big of a deal as the front
page is only generated once every 10 minutes or so (and with the changes
I'm working on today that timeout will be variable)
- John
On Sun, 2007-01-07 at 14:57 +, Robert Fitzsimons wrote:
> > Some more data on how git aff
On Sun, 7 Jan 2007 11:50:57 +0100 (MET) Jan Engelhardt wrote:
>
> On Jan 7 2007 10:03, Willy Tarreau wrote:
> >On Sun, Jan 07, 2007 at 12:58:38AM -0800, H. Peter Anvin wrote:
> >> >[..]
> >> >entries in directories with millions of files on disk. I'm not
> >> >certain it would be that easy to try
On Sun, 7 Jan 2007, Christoph Hellwig wrote:
>
> On Sun, Jan 07, 2007 at 10:03:36AM +0100, Willy Tarreau wrote:
> > The problem is that I have no sufficient FS knowledge to argument why
> > it helps here. It was a desperate attempt to fix the problem for us
> > and it definitely worked well.
>
>
"H. Peter Anvin" <[EMAIL PROTECTED]> writes:
> During extremely high load, it appears that what slows kernel.org down
> more than anything else is the time that each individual getdents()
> call takes. When I've looked this I've observed times from 200 ms to
> almost 2 seconds! Since an unpacked
> Some more data on how git affects kernel.org...
I have a quick question about the gitweb configuration, does the
$projects_list config entry point to a directory or a file?
When it is a directory gitweb ends up doing the equivalent of a 'find
$project_list' to find all the available projects, s
On Jan 7 2007 10:03, Willy Tarreau wrote:
>On Sun, Jan 07, 2007 at 12:58:38AM -0800, H. Peter Anvin wrote:
>> >[..]
>> >entries in directories with millions of files on disk. I'm not
>> >certain it would be that easy to try other filesystems on
>> >kernel.org though :-/
>>
>> Changing filesystems
On Sun, Jan 07, 2007 at 10:28:53AM +, Christoph Hellwig wrote:
> On Sun, Jan 07, 2007 at 10:03:36AM +0100, Willy Tarreau wrote:
> > The problem is that I have no sufficient FS knowledge to argument why
> > it helps here. It was a desperate attempt to fix the problem for us
> > and it definitely
On Sun, Jan 07, 2007 at 10:03:36AM +0100, Willy Tarreau wrote:
> The problem is that I have no sufficient FS knowledge to argument why
> it helps here. It was a desperate attempt to fix the problem for us
> and it definitely worked well.
XFS does rather efficient btree directories, and it does sop
On 01/07/2007 10:15 AM, Andrew Morton wrote:
Yeah, slowly-growing directories will get splattered all over the
disk.
Possible short-term fixes would be to just allocate up to (say) eight
blocks when we grow a directory by one block. Or teach the
directory-growth code to use ext3 reservations
On Sun, 7 Jan 2007 09:55:26 +0100
Willy Tarreau <[EMAIL PROTECTED]> wrote:
> On Sat, Jan 06, 2007 at 09:39:42PM -0800, Linus Torvalds wrote:
> >
> >
> > On Sat, 6 Jan 2007, H. Peter Anvin wrote:
> > >
> > > During extremely high load, it appears that what slows kernel.org down
> > > more
> > >
On Sun, Jan 07, 2007 at 12:58:38AM -0800, H. Peter Anvin wrote:
> Willy Tarreau wrote:
> >
> >At work, we had the same problem on a file server with ext3. We use rsync
> >to make backups to a local IDE disk, and we noticed that getdents() took
> >about the same time as Peter reports (0.2 to 2 secon
Willy Tarreau wrote:
At work, we had the same problem on a file server with ext3. We use rsync
to make backups to a local IDE disk, and we noticed that getdents() took
about the same time as Peter reports (0.2 to 2 seconds), especially in
maildir directories. We tried many things to fix it with
On Sat, Jan 06, 2007 at 09:39:42PM -0800, Linus Torvalds wrote:
>
>
> On Sat, 6 Jan 2007, H. Peter Anvin wrote:
> >
> > During extremely high load, it appears that what slows kernel.org down more
> > than anything else is the time that each individual getdents() call takes.
> > When I've looked
On Sat, 6 Jan 2007, H. Peter Anvin wrote:
>
> During extremely high load, it appears that what slows kernel.org down more
> than anything else is the time that each individual getdents() call takes.
> When I've looked this I've observed times from 200 ms to almost 2 seconds!
> Since an unpacked
37 matches
Mail list logo
