With the recent snubs at the engine speed :), I've been doing some profiling to find out when things are fast and when things are slow.
I've done some easy and quick minor speedups, but here are my conclusions. The numbers are later: Filters take time, but not as much as we had previous thought. There is room for them to be optimized more. It seems the real speed issue is in the zip compress driver (as someone recently mentioned here). The difference between iterating the KJV (heavily tagged OSIS zText) and GerLut (reasonably tagged GBF RawText) without filters involved, was about 5x speed difference (12.9s vs. 2.7s). Difference between KJV with and without filters does not even double the time (12.9s vs. 20s). Now, if we were using a RawText driver and getting a 2.7 iteration time without filter, then adding 8 seconds for filtering would be a significant speed hit (~3x), but we're not. So, basically, in our current configurations using a compressed driver, if we only concentrate improvement efforts on filtering (soText; which is good and can potentially removed 8 seconds of processing on a desktop) we won't even cut the iteration time in half. So, we can tackle this on two fronts. I will have a look at the compression driver and see if there is anything I can do to make things zing in there. Here are the tests boiled down: [EMAIL PROTECTED] cmdline]$ time ./outplain GerLut > gerlut.txt real 0m2.707s user 0m2.082s sys 0m0.557s [EMAIL PROTECTED] cmdline]$ time ./outplain KJV > kjv.txt real 0m12.900s user 0m11.947s sys 0m0.623s ADDED SWBASICFILTER loop back in [EMAIL PROTECTED] cmdline]$ time ./outplain KJV > kjv.txt real 0m13.250s user 0m12.344s sys 0m0.723s ADDED OSISPlain back in [EMAIL PROTECTED] cmdline]$ time ./outplain KJV > kjv.txt real 0m16.638s user 0m15.614s sys 0m0.824s ADDED ALL KJV FILTERS BACK IN [EMAIL PROTECTED] cmdline]$ time ./outplain KJV > kjv.txt real 0m20.055s user 0m19.068s sys 0m0.772s _______________________________________________ sword-devel mailing list: sword-devel@crosswire.org http://www.crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
