More disc cache improvements
Further to my previous efforts I have made an attempt to improve the disc cache performance even more. I would again be grateful if suitably interested users could try test CI build 2771 or later. The previous changes switched to using a small number of large files to hold all the "small" entries within the cache instead of them using individual files. This drastically improved the cache performance on several RISC OS machines. The new update extended this approach so that these block files get created with their maximum size instead of being grown every time a new cache entry is added. This change should be beneficial to RISC OS users as filecore is (apparently) dreadful at this kind of usage pattern. I have also adjusted how the computation of low bandwidth is made to try and make it more stable and less "trigger happy". I would be interested in getting the details from using the updated test release in the same way as previously detailed. (The overall bandwidth message from the log) I am especially interested in testing from the Iyonix as this was right on the edge of usefulness previously. It is probably not worth bothering with the RPi and ARM mini platforms as their reported bandwidth last time was awful and I do not envisage an improvement of that many orders of magnitude. -- Regards Vincent http://www.kyllikki.org/
Re: More disc cache improvements
In article <20150505103028.gg19...@kyllikki.org>, Vincent Sanders wrote: > This change should be beneficial to RISC OS users as filecore is > (apparently) dreadful at this kind of usage pattern. I wonder if this is because RISC OS files are 'defragmented' all the time - I think I am correct in saying that when a file becomes too big for the contiguous space it currently occupies, it is copied into a larger block of free space. This means that as the file grows, it may be regularly being copied somewhere else on the disc, and with files that may be hundreds of MB in size, this will be slow - very much so when the drive is an SD card. -- Chris Johnson
Re: More disc cache improvements
On Tue, May 05, 2015 at 11:48:46AM +0100, cj wrote: > In article <20150505103028.gg19...@kyllikki.org>, >Vincent Sanders wrote: > > This change should be beneficial to RISC OS users as filecore is > > (apparently) dreadful at this kind of usage pattern. > > I wonder if this is because RISC OS files are 'defragmented' all the > time - I think I am correct in saying that when a file becomes too > big for the contiguous space it currently occupies, it is copied into > a larger block of free space. This means that as the file grows, it > may be regularly being copied somewhere else on the disc, and with > files that may be hundreds of MB in size, this will be slow - very > much so when the drive is an SD card. Ish. FileCore has supported fragmented files for a long time (ISTR E implements this). It was only earlier versions that didn't support fragmented files and thus had to rearrange on file growth. B.
Re: More disc cache improvements
In article <20150505103028.gg19...@kyllikki.org>, Vincent Sanders wrote: > Further to my previous efforts I have made an attempt to improve the > disc cache performance even more. I would again be grateful if > suitably interested users could try test CI build 2771 or later. [snip] > I would be interested in getting the details from using the updated > test release in the same way as previously detailed. (The overall > bandwidth message from the log) OK. VRPC, 4.02, content/llcache.c llcache_finalise 3361: Backing store average bandwidth 15983181 bytes/second Hope that this is useful. > I am especially interested in testing from the Iyonix as this was > right on the edge of usefulness previously. It is probably not worth > bothering with the RPi and ARM mini platforms as their reported > bandwidth last time was awful and I do not envisage an improvement of > that many orders of magnitude.
Re: More disc cache improvements
In message <20150505103028.gg19...@kyllikki.org> on 5 May 2015 Vincent Sanders wrote: > Further to my previous efforts I have made an attempt to improve the > disc cache performance even more. I would again be grateful if > suitably interested users could try test CI build 2771 or later. > The previous changes switched to using a small number of large files > to hold all the "small" entries within the cache instead of them using > individual files. This drastically improved the cache performance on > several RISC OS machines. > The new update extended this approach so that these block files get > created with their maximum size instead of being grown every time a > new cache entry is added. > This change should be beneficial to RISC OS users as filecore is > (apparently) dreadful at this kind of usage pattern. > I have also adjusted how the computation of low bandwidth is made to > try and make it more stable and less "trigger happy". > I would be interested in getting the details from using the updated > test release in the same way as previously detailed. (The overall > bandwidth message from the log) > I am especially interested in testing from the Iyonix as this was > right on the edge of usefulness previously. It is probably not worth > bothering with the RPi and ARM mini platforms as their reported > bandwidth last time was awful and I do not envisage an improvement of > that many orders of magnitude. As you expected, no benefit for ARMini users: ARMini with RO 5.20 (10 June 2013) and !Boot on the SD card, not the USB drive. NS #2771 There was no complaint about the bandwidth being too low but the reported average from going only to the www.buxtonweather.co.uk site was 27269 bytes/second. The previous figure I reported was 33092 bytes/second (NS#2697). Regards Andrew -- Andrew Pinder
Re: More disc cache improvements
In article <20150505103028.gg19...@kyllikki.org>, Vincent Sanders wrote: > I am especially interested in testing from the Iyonix as this was > right on the edge of usefulness previously. Just loaded a few random pages from BBC and from ROOL on the Iyonix. I did have the disc cache disabled before, because of bandwidth warnings etc. Cache now set to 1 GB. This is the result: (868.07) content/llcache.c llcache_finalise 3361: Backing store average bandwidth 652836 bytes/second This is certainly better than the last time I tried this test, probably getting on for about double the bandwidth. I'll try on the PB later. -- Chris Johnson
Re: More disc cache improvements
Vincent Sanders, on 5 May, wrote: > Further to my previous efforts I have made an attempt to improve the disc > cache performance even more. I would again be grateful if suitably > interested users could try test CI build 2771 or later. [snip] > I am especially interested in testing from the Iyonix as this was right on > the edge of usefulness previously. [snip] NetSurf #2771 >From the Iyonix on heavy duty news sites, 347744, 375645, 366788 and 537769 bytes/sec. Lighter sites with presumably less to cache returned 56655 and 44821 bytes/sec, but the "too slow" warning never appeared. On VRPC OS4.39 on Windows 7, 508804, 1033119 and 1119900 bytes/sec which is better than previously. -- David Pitt
Re: More disc cache improvements
Using a PandaBoard with the cache on a Fat32 formatted SSD (not the SD card) gave the following: (663.19) content/llcache.c llcache_finalise 3361: Backing store average bandwidth 414186 bytes/second -- Chris Johnson
