Disk I/O as a bottleneck?
I have a PC with powerful processor, lots of RAM and SATA hard disk. Nevertheless I noticed that sometimes applications (evolution E-mail software and Firefox[iceweasel] Web browser) have the sluggish feel of a busy system (command line response time remains crisp, however, because the processor is 4x2 core one [4 cores, each multithreads as 2]). I run the gnome-system-monitor all the time. I notice that even when those applications feel sluggish, only one or at most two CPUs have high utilization, and there is plenty of free RAM (no swap space is used at all). Disk I/O is not monitored by gnome-system-monitor. So I suspect that the system is slowed down by disk I/O. I would like to eliminate it as a possible cause for the applications' sluggish feel. I ran smartctl tests on the hard disk, and they gave it clean bill of health. Therefore I/O error recovery should not be the reason for performance degradation. I am asking Collective Wisdom for advice about how to do: 1. Monitoring disk I/O load (counting I/O requests is not sufficient, as each request takes different time to complete due for example to disk head seeks or platter rotation time). 2. Disk scheduler fine-tuning possibilities to optimize disk I/O handling. 3. If smartctl is not sufficient to ensure that no I/O error overhead is incurred, how to better assess the hard disk's health? Thanks, --- Omer -- My Commodore 64 is suffering from slowness and insufficiency of memory; and its display device is grievously short of pixels. Can anyone help? My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
you are stepping into "never-never" land ;) "iostat -x -k 1" is your friend - just make sure you open a very wide terminal in which to look at it. disks are notoriously slow, regardless of error cases. it is enough if an applications perform a lot of random I/O - to make them work very slow. i'd refer you to the slides of the "linux I/O" lecture, at: http://haifux.org/lectures/254/alice_and_bob_in_io_land/ read them through. there are also some links to pages that discuss disk I/O tweaking. as for the elevator - you could try using the "deadline" elevator and see if this gives you any remedy. if you eventually decide that it is indeed disk I/O that slows you down, and if you have a lot of money to spend - you could consider buying an enterprise-grade SSD (e.g. from fusion I/O or from OCZ - although for your use-case, some of the cheaper SSDs will do) and use it instead of the hard disks. they only cost thousands of dollars for a 600GB SSD ;) --guy On Sat, 2011-05-07 at 15:29 +0300, Omer Zak wrote: > I have a PC with powerful processor, lots of RAM and SATA hard disk. > Nevertheless I noticed that sometimes applications (evolution E-mail > software and Firefox[iceweasel] Web browser) have the sluggish feel of a > busy system (command line response time remains crisp, however, because > the processor is 4x2 core one [4 cores, each multithreads as 2]). > > I run the gnome-system-monitor all the time. > > I notice that even when those applications feel sluggish, only one or at > most two CPUs have high utilization, and there is plenty of free RAM (no > swap space is used at all). > > Disk I/O is not monitored by gnome-system-monitor. > So I suspect that the system is slowed down by disk I/O. I would like > to eliminate it as a possible cause for the applications' sluggish feel. > > I ran smartctl tests on the hard disk, and they gave it clean bill of > health. Therefore I/O error recovery should not be the reason for > performance degradation. > > I am asking Collective Wisdom for advice about how to do: > 1. Monitoring disk I/O load (counting I/O requests is not sufficient, as > each request takes different time to complete due for example to disk > head seeks or platter rotation time). > 2. Disk scheduler fine-tuning possibilities to optimize disk I/O > handling. > 3. If smartctl is not sufficient to ensure that no I/O error overhead is > incurred, how to better assess the hard disk's health? > > Thanks, > --- Omer > ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
20th Anniversary T-Shirt Design Contest
Hi, I am designing image for annual Linux.com Store T-shirt design contest: http://www.linux.com/tshirt-design-contest The draft of the image is here: http://www.makelinux.net/art/20y/ Can you please give me your feedback to improve the image? Please note, I am not professional graphics designer. It is just hobby. Thanks -- Constantine Shulyupin http://www.MakeLinux.co.il/ Embedded Linux Systems, Device Drivers, TI DaVinci ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sat, May 7, 2011 at 4:06 PM, guy keren wrote: > > you are stepping into "never-never" land ;) > > "iostat -x -k 1" is your friend - just make sure you open a very wide > terminal in which to look at it. > > disks are notoriously slow, regardless of error cases. it is enough if > an applications perform a lot of random I/O - to make them work very > slow. > > i'd refer you to the slides of the "linux I/O" lecture, at: > > http://haifux.org/lectures/254/alice_and_bob_in_io_land/ > > read them through. there are also some links to pages that discuss disk > I/O tweaking. > > as for the elevator - you could try using the "deadline" elevator and > see if this gives you any remedy. > > if you eventually decide that it is indeed disk I/O that slows you down, > and if you have a lot of money to spend - you could consider buying an > enterprise-grade SSD (e.g. from fusion I/O or from OCZ - although for > your use-case, some of the cheaper SSDs will do) and use it instead of > the hard disks. they only cost thousands of dollars for a 600GB SSD ;) Would probably be cheaper to get a bunch of SATAs into a raid array - spindle count matters after all. My home machine is not too new, but it definitely took wing after I replaced one large SATA disk with 6 smaller ones in a raid5 (I'm not risky enough for raid0) > > --guy > > On Sat, 2011-05-07 at 15:29 +0300, Omer Zak wrote: >> I have a PC with powerful processor, lots of RAM and SATA hard disk. >> Nevertheless I noticed that sometimes applications (evolution E-mail >> software and Firefox[iceweasel] Web browser) have the sluggish feel of a >> busy system (command line response time remains crisp, however, because >> the processor is 4x2 core one [4 cores, each multithreads as 2]). >> >> I run the gnome-system-monitor all the time. >> >> I notice that even when those applications feel sluggish, only one or at >> most two CPUs have high utilization, and there is plenty of free RAM (no >> swap space is used at all). >> >> Disk I/O is not monitored by gnome-system-monitor. >> So I suspect that the system is slowed down by disk I/O. I would like >> to eliminate it as a possible cause for the applications' sluggish feel. >> >> I ran smartctl tests on the hard disk, and they gave it clean bill of >> health. Therefore I/O error recovery should not be the reason for >> performance degradation. >> >> I am asking Collective Wisdom for advice about how to do: >> 1. Monitoring disk I/O load (counting I/O requests is not sufficient, as >> each request takes different time to complete due for example to disk >> head seeks or platter rotation time). >> 2. Disk scheduler fine-tuning possibilities to optimize disk I/O >> handling. >> 3. If smartctl is not sufficient to ensure that no I/O error overhead is >> incurred, how to better assess the hard disk's health? >> >> Thanks, >> --- Omer >> > > > > ___ > Linux-il mailing list > Linux-il@cs.huji.ac.il > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sat, 2011-05-07 at 16:19 +0300, Dima (Dan) Yasny wrote: > On Sat, May 7, 2011 at 4:06 PM, guy keren wrote: > > > > you are stepping into "never-never" land ;) > > > > "iostat -x -k 1" is your friend - just make sure you open a very wide > > terminal in which to look at it. > > > > disks are notoriously slow, regardless of error cases. it is enough if > > an applications perform a lot of random I/O - to make them work very > > slow. > > > > i'd refer you to the slides of the "linux I/O" lecture, at: > > > > http://haifux.org/lectures/254/alice_and_bob_in_io_land/ > > > > read them through. there are also some links to pages that discuss disk > > I/O tweaking. > > > > as for the elevator - you could try using the "deadline" elevator and > > see if this gives you any remedy. > > > > if you eventually decide that it is indeed disk I/O that slows you down, > > and if you have a lot of money to spend - you could consider buying an > > enterprise-grade SSD (e.g. from fusion I/O or from OCZ - although for > > your use-case, some of the cheaper SSDs will do) and use it instead of > > the hard disks. they only cost thousands of dollars for a 600GB SSD ;) > > Would probably be cheaper to get a bunch of SATAs into a raid array - > spindle count matters after all. > > My home machine is not too new, but it definitely took wing after I > replaced one large SATA disk with 6 smaller ones in a raid5 (I'm not > risky enough for raid0) > you are, of-course, quite right. provided that a hardware RAID controller is being used. --guy > > > > --guy > > > > On Sat, 2011-05-07 at 15:29 +0300, Omer Zak wrote: > >> I have a PC with powerful processor, lots of RAM and SATA hard disk. > >> Nevertheless I noticed that sometimes applications (evolution E-mail > >> software and Firefox[iceweasel] Web browser) have the sluggish feel of a > >> busy system (command line response time remains crisp, however, because > >> the processor is 4x2 core one [4 cores, each multithreads as 2]). > >> > >> I run the gnome-system-monitor all the time. > >> > >> I notice that even when those applications feel sluggish, only one or at > >> most two CPUs have high utilization, and there is plenty of free RAM (no > >> swap space is used at all). > >> > >> Disk I/O is not monitored by gnome-system-monitor. > >> So I suspect that the system is slowed down by disk I/O. I would like > >> to eliminate it as a possible cause for the applications' sluggish feel. > >> > >> I ran smartctl tests on the hard disk, and they gave it clean bill of > >> health. Therefore I/O error recovery should not be the reason for > >> performance degradation. > >> > >> I am asking Collective Wisdom for advice about how to do: > >> 1. Monitoring disk I/O load (counting I/O requests is not sufficient, as > >> each request takes different time to complete due for example to disk > >> head seeks or platter rotation time). > >> 2. Disk scheduler fine-tuning possibilities to optimize disk I/O > >> handling. > >> 3. If smartctl is not sufficient to ensure that no I/O error overhead is > >> incurred, how to better assess the hard disk's health? > >> > >> Thanks, > >> --- Omer > >> > > > > > > > > ___ > > Linux-il mailing list > > Linux-il@cs.huji.ac.il > > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il > > ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sat, May 7, 2011 at 4:41 PM, guy keren wrote: > On Sat, 2011-05-07 at 16:19 +0300, Dima (Dan) Yasny wrote: >> On Sat, May 7, 2011 at 4:06 PM, guy keren wrote: >> > >> > you are stepping into "never-never" land ;) >> > >> > "iostat -x -k 1" is your friend - just make sure you open a very wide >> > terminal in which to look at it. >> > >> > disks are notoriously slow, regardless of error cases. it is enough if >> > an applications perform a lot of random I/O - to make them work very >> > slow. >> > >> > i'd refer you to the slides of the "linux I/O" lecture, at: >> > >> > http://haifux.org/lectures/254/alice_and_bob_in_io_land/ >> > >> > read them through. there are also some links to pages that discuss disk >> > I/O tweaking. >> > >> > as for the elevator - you could try using the "deadline" elevator and >> > see if this gives you any remedy. >> > >> > if you eventually decide that it is indeed disk I/O that slows you down, >> > and if you have a lot of money to spend - you could consider buying an >> > enterprise-grade SSD (e.g. from fusion I/O or from OCZ - although for >> > your use-case, some of the cheaper SSDs will do) and use it instead of >> > the hard disks. they only cost thousands of dollars for a 600GB SSD ;) >> >> Would probably be cheaper to get a bunch of SATAs into a raid array - >> spindle count matters after all. >> >> My home machine is not too new, but it definitely took wing after I >> replaced one large SATA disk with 6 smaller ones in a raid5 (I'm not >> risky enough for raid0) >> > > you are, of-course, quite right. provided that a hardware RAID > controller is being used. > I've seen performance increases even using fakeraid (the feared intel matrix 8.x in my machine) and mdadm. Especially in a machine that has a UPS (to step in instead of a BBU) and, like mentioned above - lots of crunchpower > --guy > > >> > >> > --guy >> > >> > On Sat, 2011-05-07 at 15:29 +0300, Omer Zak wrote: >> >> I have a PC with powerful processor, lots of RAM and SATA hard disk. >> >> Nevertheless I noticed that sometimes applications (evolution E-mail >> >> software and Firefox[iceweasel] Web browser) have the sluggish feel of a >> >> busy system (command line response time remains crisp, however, because >> >> the processor is 4x2 core one [4 cores, each multithreads as 2]). >> >> >> >> I run the gnome-system-monitor all the time. >> >> >> >> I notice that even when those applications feel sluggish, only one or at >> >> most two CPUs have high utilization, and there is plenty of free RAM (no >> >> swap space is used at all). >> >> >> >> Disk I/O is not monitored by gnome-system-monitor. >> >> So I suspect that the system is slowed down by disk I/O. I would like >> >> to eliminate it as a possible cause for the applications' sluggish feel. >> >> >> >> I ran smartctl tests on the hard disk, and they gave it clean bill of >> >> health. Therefore I/O error recovery should not be the reason for >> >> performance degradation. >> >> >> >> I am asking Collective Wisdom for advice about how to do: >> >> 1. Monitoring disk I/O load (counting I/O requests is not sufficient, as >> >> each request takes different time to complete due for example to disk >> >> head seeks or platter rotation time). >> >> 2. Disk scheduler fine-tuning possibilities to optimize disk I/O >> >> handling. >> >> 3. If smartctl is not sufficient to ensure that no I/O error overhead is >> >> incurred, how to better assess the hard disk's health? >> >> >> >> Thanks, >> >> --- Omer >> >> >> > >> > >> > >> > ___ >> > Linux-il mailing list >> > Linux-il@cs.huji.ac.il >> > http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il >> > > > > ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: 20th Anniversary T-Shirt Design Contest
Hi Constantine, first of all, thanks for your effort. I'm commenting what I feel about your image in public - I hope it's OK. Good luck in the competition. On Saturday 07 May 2011 16:02:35 Constantine Shulyupin wrote: > Hi, > > I am designing image for annual Linux.com Store T-shirt design > contest: http://www.linux.com/tshirt-design-contest > > The draft of the image is here: http://www.makelinux.net/art/20y/ > > Can you please give me your feedback to improve the image? > Please note, I am not professional graphics designer. It is just hobby. > Well, for professional vs. amateur see: http://www.paulgraham.com/opensource.html An amateur used to mean someone who does what they do for love of their art/craft/science/etc. instead (or in addition to) money, and was actually a compliment. I consider myself an amateur for many fields, and while a professional is often a good idea, the open-minded professionals can still sometimes learn from amateurs. I asked someone with a Ph.D. in psychotherapy and a lot of experience in dealing with patients, and he agreed with me that there are some things he can still learn from people with little experience in psychotherapy. That's the way it is. Well, back to the image, and I'm not an expert graphics designer either: 1. In general, I think it's nice. I don't think I could have ever created such a well-drawn aesthetic penguin myself. :-) (At least not using a mouse, which I find unsuitable for that.). 2. I think the feet of the penguin are a bit too unrealistic. 3. I don't like the fade-to-white gradients - they obstruct recognising the view. Maybe try a different gradient, perhaps from one colour to the other. 4. "20Y" is too obscure. Maybe say "20" with a smaller years. 5. I find the Penguin too unrealistic, cartoony and it seems to generate a negative impression in me. Maybe try making it more positive. The original Tux the penguin image by Larry Ewing ( http://en.wikipedia.org/wiki/Tux ) is cute and adorable and make us emphasise with Linux and identify with it. These are the things off the top of my head. Shabath Shalom, and enjoy the Memorial Day and the Israeli Independence Day. Regards, Shlomi Fish P.S: in a self-promotion, you may wish to take a look at http://www.shlomifish.org/humour/Selena-Mandrake/ , which is a screenplay I've begun writing that aims to be a supernatural dramedie, and one of my most farfetched pieces yet. I've only written the first few scenes, but I have a solid idea for more stuff there. Furthermore, if you have a kindle, you may wish to buy the Kindle version of "The Enemy and How I Helped to Fight it": http://www.amazon.com/Enemy-How-Helped-Fight-ebook/dp/B004YTSWS0/ref=sr_1_1 The price is higher than what I wanted, but if you want, you can read the story on my site, and just comment about it for future generations (and maybe make a smaller donation to my PayPal account). -- - Shlomi Fish http://www.shlomifish.org/ My Aphorisms - http://www.shlomifish.org/humour.html C++ is complex, complexifying and complexified. (With apologies to the Oxford English Dictionary). Please reply to list if it's a mailing list post - http://shlom.in/reply . ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
I would suggest making the check I mention in my own blog, in particular if you're running an old kernel. There has been a bug in the way the kernel handles heavy disk loads. http://billauer.co.il/blog/2010/10/disk-io-scheduler-load-dd-freeze-stall-hang/ Omer Zak wrote: I have a PC with powerful processor, lots of RAM and SATA hard disk. Nevertheless I noticed that sometimes applications (evolution E-mail software and Firefox[iceweasel] Web browser) have the sluggish feel of a busy system (command line response time remains crisp, however, because the processor is 4x2 core one [4 cores, each multithreads as 2]). I run the gnome-system-monitor all the time. I notice that even when those applications feel sluggish, only one or at most two CPUs have high utilization, and there is plenty of free RAM (no swap space is used at all). Disk I/O is not monitored by gnome-system-monitor. So I suspect that the system is slowed down by disk I/O. I would like to eliminate it as a possible cause for the applications' sluggish feel. I ran smartctl tests on the hard disk, and they gave it clean bill of health. Therefore I/O error recovery should not be the reason for performance degradation. I am asking Collective Wisdom for advice about how to do: 1. Monitoring disk I/O load (counting I/O requests is not sufficient, as each request takes different time to complete due for example to disk head seeks or platter rotation time). 2. Disk scheduler fine-tuning possibilities to optimize disk I/O handling. 3. If smartctl is not sufficient to ensure that no I/O error overhead is incurred, how to better assess the hard disk's health? Thanks, --- Omer -- Web: http://www.billauer.co.il ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
Hi Omer, On Sat, May 7, 2011 at 3:29 PM, Omer Zak wrote: > I have a PC with powerful processor, lots of RAM and SATA hard disk. > Nevertheless I noticed that sometimes applications (evolution E-mail > software and Firefox[iceweasel] Web browser) have the sluggish feel of a > busy system (command line response time remains crisp, however, because > the processor is 4x2 core one [4 cores, each multithreads as 2]). > > I run the gnome-system-monitor all the time. > > I notice that even when those applications feel sluggish, only one or at > most two CPUs have high utilization, and there is plenty of free RAM (no > swap space is used at all). > The phenomena you are describing could be (though other explanations are still possible) a result of a completely different issue: Having 4 cores with 2 virtual cores on each means 8 threads could, theoretically, run in parallel. However, there is a large number of factors preventing 4 cores CPU from being 4 times faster than a single core CPU. The most common are, as of time being, the applications themselves - many of them are not written in a manner which enables them to use more than 1 CPU. Usually, it is either due to doing all of the CPU intensive work in one thread (i.e. the thread which renders the webpages to the screen in iceweasel) or due to coarse grain locking (i.e. locking the entire mozilla application every time we do garbage collection on the java-script heap, etc.). There are also issues of resource contention (i.e. firefox has a big working set, and therefore its speed is more affected by the amount of time it takes to access the RAM than it is affected by the CPU power - the CPU tends to mostly wait for data to arrive from the RAM), but that is less likely the case according to your description. When large amount of RAM is available, the chances that the sluggishness is due to slow disk access are rather slow - Linux is very aggressive about keeping everything in the cache, and if your application was spending most of its time waiting for disk I/O, you wouldn't have seen large CPU usage percentages - it was going into a "sleep" state until the data arrives, and wouldn't affect the user CPU usage percentage. To verify that, run top while the application is sluggish, and check if it the CPU usage is mostly in the user code (pressing "1" while top is running will show the per-CPU statistics in the top half of the screen, the percentage next to "us" is the relative amount of time the CPU spent running user code since the last sample). Good luck in hunting down your slowness. --Shachar ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: 20th Anniversary T-Shirt Design Contest
Thank you, Shlomi. I've updated the image ( http://www.makelinux.net/art/20y/ ) On Sat, May 7, 2011 at 4:56 PM, Shlomi Fish wrote: > Hi Constantine, > > first of all, thanks for your effort. I'm commenting what I feel about your > image in public - I hope it's OK. Good luck in the competition. > > On Saturday 07 May 2011 16:02:35 Constantine Shulyupin wrote: >> Hi, >> >> I am designing image for annual Linux.com Store T-shirt design >> contest: http://www.linux.com/tshirt-design-contest >> >> The draft of the image is here: http://www.makelinux.net/art/20y/ >> >> Can you please give me your feedback to improve the image? >> Please note, I am not professional graphics designer. It is just hobby. >> > > Well, for professional vs. amateur see: > > http://www.paulgraham.com/opensource.html > > An amateur used to mean someone who does what they do for love of their > art/craft/science/etc. instead (or in addition to) money, and was actually a > compliment. I consider myself an amateur for many fields, and while a > professional is often a good idea, the open-minded professionals can still > sometimes learn from amateurs. I asked someone with a Ph.D. in psychotherapy > and a lot of experience in dealing with patients, and he agreed with me that > there are some things he can still learn from people with little experience in > psychotherapy. That's the way it is. > > Well, back to the image, and I'm not an expert graphics designer either: > > 1. In general, I think it's nice. I don't think I could have ever created such > a well-drawn aesthetic penguin myself. :-) (At least not using a mouse, which > I find unsuitable for that.). > > 2. I think the feet of the penguin are a bit too unrealistic. > > 3. I don't like the fade-to-white gradients - they obstruct recognising the > view. Maybe try a different gradient, perhaps from one colour to the other. > > 4. "20Y" is too obscure. Maybe say "20" with a smaller years. > > 5. I find the Penguin too unrealistic, cartoony and it seems to generate a > negative impression in me. Maybe try making it more positive. The original Tux > the penguin image by Larry Ewing ( http://en.wikipedia.org/wiki/Tux ) is cute > and adorable and make us emphasise with Linux and identify with it. > > > > These are the things off the top of my head. > > Shabath Shalom, and enjoy the Memorial Day and the Israeli Independence Day. > > Regards, > > Shlomi Fish > > P.S: in a self-promotion, you may wish to take a look at > http://www.shlomifish.org/humour/Selena-Mandrake/ , which is a screenplay I've > begun writing that aims to be a supernatural dramedie, and one of my most > farfetched pieces yet. I've only written the first few scenes, but I have a > solid idea for more stuff there. > > Furthermore, if you have a kindle, you may wish to buy the Kindle version of > "The Enemy and How I Helped to Fight it": > > http://www.amazon.com/Enemy-How-Helped-Fight-ebook/dp/B004YTSWS0/ref=sr_1_1 > > The price is higher than what I wanted, but if you want, you can read the > story on my site, and just comment about it for future generations (and maybe > make a smaller donation to my PayPal account). > > -- > - > Shlomi Fish http://www.shlomifish.org/ > My Aphorisms - http://www.shlomifish.org/humour.html > > C++ is complex, complexifying and complexified. > (With apologies to the Oxford English Dictionary). > > Please reply to list if it's a mailing list post - http://shlom.in/reply . > -- Constantine Shulyupin http://www.MakeLinux.co.il/ Embedded Linux Systems, Device Drivers, TI DaVinci ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sat, May 7, 2011 at 4:06 PM, guy keren wrote: > > if you eventually decide that it is indeed disk I/O that slows you down, > and if you have a lot of money to spend - you could consider buying an > enterprise-grade SSD (e.g. from fusion I/O or from OCZ - although for > your use-case, some of the cheaper SSDs will do) and use it instead of > the hard disks. they only cost thousands of dollars for a 600GB SSD ;) > Is there a reason you're recommending such an expensive drives? I thought some time ago to buy a "regular" 40-80Gb and install the OS+swap there, and have a "regular" drive around for the rest of the data. Is there a reason this won't work? ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sat, 2011-05-07 at 21:49 +0300, Elazar Leibovich wrote: > On Sat, May 7, 2011 at 4:06 PM, guy keren wrote: > > if you eventually decide that it is indeed disk I/O that slows > you down, > and if you have a lot of money to spend - you could consider > buying an > enterprise-grade SSD (e.g. from fusion I/O or from OCZ - > although for > your use-case, some of the cheaper SSDs will do) and use it > instead of > the hard disks. they only cost thousands of dollars for a > 600GB SSD ;) > > > Is there a reason you're recommending such an expensive drives? > I thought some time ago to buy a "regular" 40-80Gb and install the OS > +swap there, and have a "regular" drive around for the rest of the > data. Is there a reason this won't work? I suspect that speeding up /usr won't help improve performance that much. The applications, which seem to be sluggish, deal with a lot of user data in /home. Furthermore, this user data varies a lot with time, hence it is not that good idea to store it in SSD. I liked more the idea of using a RAID scheme. --- Omer -- My Commodore 64 is suffering from slowness and insufficiency of memory; and its display device is grievously short of pixels. Can anyone help? My own blog is at http://www.zak.co.il/tddpirate/ My opinions, as expressed in this E-mail message, are mine alone. They do not represent the official policy of any organization with which I may be affiliated in any way. WARNING TO SPAMMERS: at http://www.zak.co.il/spamwarning.html ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sat, 2011-05-07 at 21:49 +0300, Elazar Leibovich wrote: > On Sat, May 7, 2011 at 4:06 PM, guy keren wrote: > > if you eventually decide that it is indeed disk I/O that slows > you down, > and if you have a lot of money to spend - you could consider > buying an > enterprise-grade SSD (e.g. from fusion I/O or from OCZ - > although for > your use-case, some of the cheaper SSDs will do) and use it > instead of > the hard disks. they only cost thousands of dollars for a > 600GB SSD ;) > > > Is there a reason you're recommending such an expensive drives? > I thought some time ago to buy a "regular" 40-80Gb and install the OS > +swap there, and have a "regular" drive around for the rest of the > data. Is there a reason this won't work? are you talking about using a low-end SSD? the problem with them, is that often their throughput for sequential operations is lower then that of normal hard disks. or are you talking about something different? --guy ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sun, May 8, 2011 at 12:20 AM, guy keren wrote: > > are you talking about using a low-end SSD? > I'm actually not a big SSD expert, but I'm talking about relatively cheap SSD you can find in Ivory/Ksp, for instance Intel's http://www.zap.co.il/model.aspx?modelid=751136 > > the problem with them, is that often their throughput for sequential > operations is lower then that of normal hard disks. > Yeah, but what matters for the average user's computer speed is the random access speed, even if copying the 1Gb file will be a bit slower, when using the computer it'll be much faster, wouldn't it? ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sun, 2011-05-08 at 00:21 +0300, Elazar Leibovich wrote: > > > On Sun, May 8, 2011 at 12:20 AM, guy keren wrote: > > > are you talking about using a low-end SSD? > > > I'm actually not a big SSD expert, but I'm talking about relatively > cheap SSD you can find in Ivory/Ksp, for instance Intel's > http://www.zap.co.il/model.aspx?modelid=751136 > > > the problem with them, is that often their throughput for > sequential > operations is lower then that of normal hard disks. > > > Yeah, but what matters for the average user's computer speed is the > random access speed, even if copying the 1Gb file will be a bit > slower, when using the computer it'll be much faster, wouldn't it? i guess the answer will be "it depends" :0 the fact is that a desktop user still does a lot of sequential I/O - so the sequential I/O speed still matters. another thing to note - the SSDs tend to start performing much worse if you fill them up to their max capacity. better use them in a lower capacity (e.g. 70-80% fill-factor), to keep their performance sane. i suggest that, once you get this drive, you come and tell us if you feel an improvement. then, once year after that - tell us again. --guy ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sat, May 07, 2011, guy keren wrote about "Re: Disk I/O as a bottleneck?": > and if you have a lot of money to spend - you could consider buying an > enterprise-grade SSD (e.g. from fusion I/O or from OCZ - although for > your use-case, some of the cheaper SSDs will do) and use it instead of > the hard disks. they only cost thousands of dollars for a 600GB SSD ;) Instead of buying a huge SSD for "thousands of dollars" another option you might consider is to buy a relatively small SSD with just enough space to hold your "/" partition and swap space. Even 20 G may be enough. The rest of your disk - holding your source code, photos, songs, movies, or whatever you typically fill a terabyte with, will be a normal, cheap, hard disk. Several of my friends have gone with such a setup on their latest computer, and they are very pleased. -- Nadav Har'El|Sunday, May 8 2011, 4 Iyyar 5771 n...@math.technion.ac.il |- Phone +972-523-790466, ICQ 13349191 |My password is my dog's name. His name is http://nadav.harel.org.il |a#j!4@h, but I change it every month. ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: 20th Anniversary T-Shirt Design Contest
Hi Const, On Saturday 07 May 2011 20:09:42 Constantine Shulyupin wrote: > Thank you, Shlomi. I've updated the image ( > http://www.makelinux.net/art/20y/ ) > Nice, thanks. I'll be off the computer for a few days, but I'll take a look after that. I hope the other members of the list give you some other useful input. Regards, Shlomi Fish -- - Shlomi Fish http://www.shlomifish.org/ Apple Inc. is Evil - http://www.shlomifish.org/open-source/anti/apple/ buu: do you have a functional spec? An architecture document? An interface whitepaper? A developer's guide? A user manual? A "The BL-Book" and "BL - The Program"? rindolf: no, no, no no and no Please reply to list if it's a mailing list post - http://shlom.in/reply . ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sun, 08 May 2011 07:28:49 +0300 Nadav Har'El wrote: > Instead of buying a huge SSD for "thousands of dollars" another option you > might consider is to buy a relatively small SSD with just enough space to > hold your "/" partition and swap space. Even 20 G may be enough. > The rest of your disk - holding your source code, photos, songs, movies, > or whatever you typically fill a terabyte with, will be a normal, cheap, > hard disk. I don't agree with this setup. Regular consumer drives setup with RAID to stripe are going to be much, much faster and have less problems in the long run than single SSDs at this point as well as being a better value until prices change a lot. Consider not using swap, because swap when in use causes a lot of thrashing and kills performance especially if you only have 1 or 2 drives. If you have a reasonably fast CPU (as the OP wrote) and more than 1G of RAM you can live without swap. Try it and like it: /sbin/swapoff -a Run for a few days and see if your performance doesn't improve. The only problem I can think of is if you run leaky code and don't have swap your system will lock up sooner. If you do have swap perhaps you will be able to see it coming. Another thing to consider is what filesystem(s) you use and what your mountpoints are. That's a religious debate and I'm not going to get into it except to say different filesystems have different advantages and disadvantages and it's worthwhile to do a bit of research to see which one(s) will work for you. You might consider one filesystem for volatile directories like /tmp and /var/log and others for relatively static filesystems like /. You should also consider your whole filesystem structure and the way you have mountpoints set up. If you have one big filesystem for everything (common but incorrect desktop config) you are not going to get the best performance possible. ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On May 8, 2011, at 7:54 AM, is...@zahav.net.il wrote: I don't agree with this setup. Regular consumer drives setup with RAID to stripe are going to be much, much faster and have less problems in the long run than single SSDs at this point as well as being a better value until prices change a lot. If it's stuff you don't use often, or use sequentially, such as videos, cd/dvd software images, etc, you may consider USB drives. Be aware that using the NFS kernel server and USB disk drives causes kernel panics, lost data, etc. You can avoid the problem using samba shares or the user space NFS server. The user space NFS server is not compatible with some things, like RSYNC (missing function support), JDownloader (I/O on download directory) and the latest version of Ubuntu 11.04's gnome GUI (I/O error on home directory full "eye candy" turned on). With it turned off, it works. Between Kravitz, Bug and Office Depot, they occasionaly have "disk wars" where they sell USB external disks very cheaply. Geoff. -- Geoffrey S. Mendelson, N3OWJ/4X1GM Those who cannot remember the past are condemned to misquote it. ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sun, May 08, 2011 at 07:28:49AM +0300, Nadav Har'El wrote:
> On Sat, May 07, 2011, guy keren wrote about "Re: Disk I/O as a bottleneck?":
> > and if you have a lot of money to spend - you could consider buying an
> > enterprise-grade SSD (e.g. from fusion I/O or from OCZ - although for
> > your use-case, some of the cheaper SSDs will do) and use it instead of
> > the hard disks. they only cost thousands of dollars for a 600GB SSD ;)
>
> Instead of buying a huge SSD for "thousands of dollars" another option you
> might consider is to buy a relatively small SSD with just enough space to
> hold your "/" partition and swap space. Even 20 G may be enough.
> The rest of your disk - holding your source code, photos, songs, movies,
> or whatever you typically fill a terabyte with, will be a normal, cheap,
> hard disk.
>
> Several of my friends have gone with such a setup on their latest computer,
> and they are very pleased.
I am considering, for my next laptop, and taking into account the fact
that most laptops do not have space for two disks but do have some kind
of flash memory slot ("card reader") - usually SD-something, to have the
OS on a (e.g.) SD card of 16 or 32 GB. I have no other experience with
such cards, so I do not know if they are considered durable enough, fast
enough - both random and sequential IO, both compared to SATA mechanical
disks and to SATA flash ones, etc. Comments are welcome :-)
--
Didi
___
Linux-il mailing list
Linux-il@cs.huji.ac.il
http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sat, May 07, 2011, Omer Zak wrote about "Re: Disk I/O as a bottleneck?": > I suspect that speeding up /usr won't help improve performance that > much. The applications, which seem to be sluggish, deal with a lot of > user data in /home. Furthermore, this user data varies a lot with time, > hence it is not that good idea to store it in SSD. As usual, "it depends on your workload" applies. In my own personal experience (and naturally, it might differ considerably from your use case), when I see "sluggish behavior" on a desktop machine, what is actually happening is that "foreground" activity, such as playing or working with video files, or such as compilation of a large project, causes a lot of other pages to be swapped out; And then, when you switch to a different application, it needs to swap pages in - either program text (code) directly from the executables, or data pages from the swap partition. So when you switch to a GUI application, suddenly it takes a second to respond to a mouse click (it needs to swap in the relevant code and data), and when you type "ls" in a shell it takes much longer than usual (both the "ls" code and the directory are not in memory). Not only does fetching all these missing pages require a lot of seeks, which are slow on hard disks, it's even worse when that other application (which is using all that user data), continues to do a lot of seeks, and "competes" with the seeks needed to fetch the missing pages. In such a case if your system files - binaries, shared libraries, and swap, would be on a separate disk, everything might feel more responsive. If that separate disk had low seek times and hight throughput, it would be especially quick to recover from swap-outs, so you might see even better interactive behavior. Like I said, several of my friends tried this setup (SSD+hard disk) and liked the improved "feel" of the system (and the faster boot :-)). I haven't tried it myself, though. -- Nadav Har'El|Sunday, May 8 2011, 4 Iyyar 5771 n...@math.technion.ac.il |- Phone +972-523-790466, ICQ 13349191 |Christopher Robin Hood steals from the http://nadav.harel.org.il |rich and gives to the Pooh. ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
Re: Disk I/O as a bottleneck?
On Sun, May 08, 2011, is...@zahav.net.il wrote about "Re: Disk I/O as a bottleneck?": > I don't agree with this setup. Regular consumer drives setup with RAID to > stripe are going to be much, much faster and have less problems in the long > run than single SSDs at this point as well as being a better value until > prices change a lot. Having two hard disks will, at best case, *double* your seek time. This is still pretty slow, isn't it? Won't an SSD, even cheap one, have a better random access read performance? > Consider not using swap, because swap when in use causes a lot of thrashing > and kills performance especially if you only have 1 or 2 drives. If you have Even without any swap space, you can have a lot of thrashing: Clean pages - program text (the code), shared libraries, memory-mapped files, and so on - are simply "forgotten" when the page cache is needed for other things, and when you get back to that other program, suddenly all its code and libraries are not in memory, and getting them back requires a lot of random-access reads from disk, and (especially if the disk is doing other things at the same time) causes the program to appear "stuck" or "sluggish". Again, I don't know if this is the sort of problem that actually bothered the OP. -- Nadav Har'El|Sunday, May 8 2011, 4 Iyyar 5771 n...@math.technion.ac.il |- Phone +972-523-790466, ICQ 13349191 |The only "intuitive" interface is the http://nadav.harel.org.il |nipple. After that, it's all learned. ___ Linux-il mailing list Linux-il@cs.huji.ac.il http://mailman.cs.huji.ac.il/mailman/listinfo/linux-il
