Re: [RFC] LZO de/compression support - take 6

2007-05-31 Thread Satyam Sharma
On 5/31/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: [...] The author (Markus Oberhumer) of LZO provided these comments for this patch: --- I've only briefly looked over it, but it's obvious that your version does not work on architechtures which do not allow unaligned access (arm, mips, ...). A

Re: [RFC] LZO de/compression support - take 6

2007-05-31 Thread Nitin Gupta
Hi, FYI... The author (Markus Oberhumer) of LZO provided these comments for this patch: --- I've only briefly looked over it, but it's obvious that your version does not work on architechtures which do not allow unaligned access (arm, mips, ...). As for further quality assurance, your version

Re: [RFC] LZO de/compression support - take 6

2007-05-30 Thread Satyam Sharma
On 5/30/07, Johannes Stezenbach <[EMAIL PROTECTED]> wrote: On Wed, May 30, 2007, Nitin Gupta wrote: > > Again, all the original code has been retained _as-is_. Whatever was > changed, has been mentioned in that detailed changelog that I post > along with patch. Just a general remark (I haven't b

Re: [RFC] LZO de/compression support - take 6

2007-05-30 Thread Johannes Stezenbach
On Wed, May 30, 2007, Nitin Gupta wrote: > > Again, all the original code has been retained _as-is_. Whatever was > changed, has been mentioned in that detailed changelog that I post > along with patch. Just a general remark (I haven't been following this thread closely): IMHO it would be _much_

Re: [RFC] LZO de/compression support - take 6

2007-05-30 Thread Nitin Gupta
On 5/30/07, Jan Engelhardt <[EMAIL PROTECTED]> wrote: On May 30 2007 11:24, Nitin Gupta wrote: >> >> It is the bytestream (ip) that is reinterpreted as uint16_t. >> And I really doubt that the LZO author has a big-endian machine, >> given the days of ubiquitous x86. > >> le16_to_cpu it is. > > B

Re: [RFC] LZO de/compression support - take 6

2007-05-30 Thread Jan Engelhardt
On May 30 2007 11:24, Nitin Gupta wrote: >> >> It is the bytestream (ip) that is reinterpreted as uint16_t. >> And I really doubt that the LZO author has a big-endian machine, >> given the days of ubiquitous x86. > >> le16_to_cpu it is. > > But then why you think it should be > le_16_cpu() -- how

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Nitin Gupta
On 5/30/07, Jan Engelhardt <[EMAIL PROTECTED]> wrote: On May 28 2007 20:04, Nitin Gupta wrote: > > * Changelog vs. original LZO: > 1) Used standard/kernel defined data types: (this eliminated _huge_ > #ifdef chunks) > lzo_bytep -> unsigned char * > lzo_uint -> size_t > lzo_xint -> size_t Is thi

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Nitin Gupta
On 5/30/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: On Tue, May 29, 2007 at 11:10:05PM +0200, Jan Engelhardt wrote: > > On May 28 2007 19:11, Adrian Bunk wrote: You completely miss the point of my question. It's about the performance improvements of the modified code that were mentioned. What yo

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Nitin Gupta
On 5/30/07, Daniel Hazelton <[EMAIL PROTECTED]> wrote: I just noticed a bug in my testbed/benchmarking code. It's fixed, but I decided to compare version 6 of the code against the *unsafe* decompressor again. The results of the three runs I've put it through after changing it to compare against t

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Daniel Hazelton
I just noticed a bug in my testbed/benchmarking code. It's fixed, but I decided to compare version 6 of the code against the *unsafe* decompressor again. The results of the three runs I've put it through after changing it to compare against the unsafe decompressor were startling. `Tiny's` compre

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Daniel Hazelton
All problems I was having with the test-bed code have been solved, and the error I was running into was, as I suspected, in the code I used to fill the buffer for the random-data test. Results of running the new benchmark (version 6 of the benchmark, version 6 of 'tinyLZO'): 1 run averages:

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Adrian Bunk
On Tue, May 29, 2007 at 11:10:05PM +0200, Jan Engelhardt wrote: > > On May 28 2007 19:11, Adrian Bunk wrote: > > > >I have not seen any explanations: > >- Why did the upstream author write the code that way? > > I guess it's along the lines of > - portability > > (note how this contradicts itsel

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Jan Engelhardt
On May 28 2007 19:11, Adrian Bunk wrote: > >I have not seen any explanations: >- Why did the upstream author write the code that way? I guess it's along the lines of - portability (note how this contradicts itself). Really. I have yet to figure out why everyone invents their own xxx32_t types, l

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Jan Engelhardt
On May 28 2007 20:04, Nitin Gupta wrote: > > * Changelog vs. original LZO: > 1) Used standard/kernel defined data types: (this eliminated _huge_ > #ifdef chunks) > lzo_bytep -> unsigned char * > lzo_uint -> size_t > lzo_xint -> size_t Is this safe (as far as compressed LZO stream is concerned) --

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Daniel Hazelton
On Tuesday 29 May 2007 16:14:34 Daniel Hazelton wrote: > On Tuesday 29 May 2007 01:58:43 Nitin Gupta wrote: > > On 5/29/07, Bret Towe <[EMAIL PROTECTED]> wrote: > > > tested this on ppc and its still good > > > > > > is there any reason to bother with a test on amd64? > > > if there is I might be a

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Daniel Hazelton
On Tuesday 29 May 2007 01:58:43 Nitin Gupta wrote: > On 5/29/07, Bret Towe <[EMAIL PROTECTED]> wrote: > > tested this on ppc and its still good > > > > is there any reason to bother with a test on amd64? > > if there is I might be able to get to it tonight > > Yes, this test is desired on 'take 6'

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Daniel Hazelton
On Tuesday 29 May 2007 08:03:55 Nitin Gupta wrote: > On 5/29/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: > > On Tue, May 29, 2007 at 09:08:27AM +0100, Michael-Luke Jones wrote: > > > On 28 May 2007, at 18:11, Adrian Bunk wrote: > > >> I have not seen any explanations: > > >> - Why did the upstream a

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Daniel Hazelton
On Tuesday 29 May 2007 01:48:29 Nitin Gupta wrote: > On 5/29/07, Bret Towe <[EMAIL PROTECTED]> wrote: > > On 5/28/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: > > > Hi, > > > > > > Attached is tester code used for testing. > > > (developed by Daniel Hazelton -- modified slightly to now use 'take 6' >

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Nitin Gupta
On 5/29/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: On Tue, May 29, 2007 at 09:08:27AM +0100, Michael-Luke Jones wrote: > On 28 May 2007, at 18:11, Adrian Bunk wrote: > >> I have not seen any explanations: >> - Why did the upstream author write the code that way? > > Apparently due to his requirem

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Adrian Bunk
On Tue, May 29, 2007 at 09:08:27AM +0100, Michael-Luke Jones wrote: > On 28 May 2007, at 18:11, Adrian Bunk wrote: > >> I have not seen any explanations: >> - Why did the upstream author write the code that way? > > Apparently due to his requirement for extreme portability. The original > code was

Re: [RFC] LZO de/compression support - take 6

2007-05-29 Thread Michael-Luke Jones
On 28 May 2007, at 18:11, Adrian Bunk wrote: I have not seen any explanations: - Why did the upstream author write the code that way? Apparently due to his requirement for extreme portability. The original code was designed to work on everything from 16-bit DOS through CRAY supercomputers

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Nitin Gupta
On 5/29/07, Bret Towe <[EMAIL PROTECTED]> wrote: tested this on ppc and its still good is there any reason to bother with a test on amd64? if there is I might be able to get to it tonight Yes, this test is desired on 'take 6' (In future I will append version to patch bz2) Thanks, Nitin - T

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Nitin Gupta
On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: On Mon, May 28, 2007 at 09:33:32PM +0530, Nitin Gupta wrote: > On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: I have not seen any explanations: - Why did the upstream author write the code that way? - Why are your changes correct? - Why

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Nitin Gupta
On 5/29/07, Bret Towe <[EMAIL PROTECTED]> wrote: On 5/28/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: > Hi, > > Attached is tester code used for testing. > (developed by Daniel Hazelton -- modified slightly to now use 'take 6' > version for 'TinyLZO') > > Cheers, > Nitin > > On 5/28/07, Nitin Gupta

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Bret Towe
On 5/28/07, Bret Towe <[EMAIL PROTECTED]> wrote: On 5/28/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: > Hi, > > This is kernel port of LZO1X-1 compressor and LZO1X decompressor (safe > version only). > > * Changes since 'take 5' (Full Changelog after this): > - Added compressor and decomrpesssor as

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Bret Towe
On 5/28/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: Hi, Attached is tester code used for testing. (developed by Daniel Hazelton -- modified slightly to now use 'take 6' version for 'TinyLZO') Cheers, Nitin On 5/28/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: > (Using tester program from Daniel) >

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Bret Towe
On 5/28/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: Hi, This is kernel port of LZO1X-1 compressor and LZO1X decompressor (safe version only). * Changes since 'take 5' (Full Changelog after this): - Added compressor and decomrpesssor as separate and hidden config options (default: n) - Cleanups:

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Daniel Hazelton
On Monday 28 May 2007 16:18:40 Daniel Hazelton wrote: > On Monday 28 May 2007 13:11:15 Adrian Bunk wrote: > > On Mon, May 28, 2007 at 09:33:32PM +0530, Nitin Gupta wrote: > > > On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: > > >... > > > > > >> - then ensure that it works correctly on all arch

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Daniel Hazelton
On Monday 28 May 2007 13:11:15 Adrian Bunk wrote: > On Mon, May 28, 2007 at 09:33:32PM +0530, Nitin Gupta wrote: > > On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: > >... > > > >> - then ensure that it works correctly on all architectures and > > > > Already tested on x86, amd64, ppc (by Bret).

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Daniel Hazelton
On Monday 28 May 2007 13:11:15 Adrian Bunk wrote: > On Mon, May 28, 2007 at 09:33:32PM +0530, Nitin Gupta wrote: > > On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: > >... > > > >> - then ensure that it works correctly on all architectures and > > > > Already tested on x86, amd64, ppc (by Bret).

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Daniel Hazelton
On Monday 28 May 2007 13:01:09 Adrian Bunk wrote: > On Mon, May 28, 2007 at 11:55:14AM -0400, Daniel Hazelton wrote: > >... > > This is my guess as well. Though performance will likely drop when I make > > the noinline macro mean something. (This may be offset by figuring out a > > way to make like

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Adrian Bunk
On Mon, May 28, 2007 at 09:33:32PM +0530, Nitin Gupta wrote: > On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: >... >> - then ensure that it works correctly on all architectures and > > Already tested on x86, amd64, ppc (by Bret). I do not have machines > from other archs available. Bret tested

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Adrian Bunk
On Mon, May 28, 2007 at 11:55:14AM -0400, Daniel Hazelton wrote: >... > This is my guess as well. Though performance will likely drop when I make the > noinline macro mean something. (This may be offset by figuring out a way to > make likely() and unlikely() also have a meaningful effect in users

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Nitin Gupta
On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: On Mon, May 28, 2007 at 08:36:44PM +0530, Nitin Gupta wrote: >... > So, before this change, it will be good if it gets merged in mainline > and tested, at least for correctness, on all supported achs. All the > while, we will have a good feeling

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Daniel Hazelton
On Monday 28 May 2007 11:47:55 Nitin Gupta wrote: > On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: > > On Mon, May 28, 2007 at 08:10:31PM +0530, Nitin Gupta wrote: > > > Hi, > > > > > > Attached is tester code used for testing. > > > (developed by Daniel Hazelton -- modified slightly to now use

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Daniel Hazelton
On Monday 28 May 2007 11:30:55 Adrian Bunk wrote: > On Mon, May 28, 2007 at 08:10:31PM +0530, Nitin Gupta wrote: > > Hi, > > > > Attached is tester code used for testing. > > (developed by Daniel Hazelton -- modified slightly to now use 'take 6' > > version for 'TinyLZO') > > > > Cheers, > > Nitin

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Nitin Gupta
On 5/28/07, Adrian Bunk <[EMAIL PROTECTED]> wrote: On Mon, May 28, 2007 at 08:10:31PM +0530, Nitin Gupta wrote: > Hi, > > Attached is tester code used for testing. > (developed by Daniel Hazelton -- modified slightly to now use 'take 6' > version for 'TinyLZO') > > Cheers, > Nitin > > On 5/28/07,

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Adrian Bunk
On Mon, May 28, 2007 at 08:36:44PM +0530, Nitin Gupta wrote: >... > So, before this change, it will be good if it gets merged in mainline > and tested, at least for correctness, on all supported achs. All the > while, we will have a good feeling that there is still a good scope > for perf improveme

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Adrian Bunk
On Mon, May 28, 2007 at 08:10:31PM +0530, Nitin Gupta wrote: > Hi, > > Attached is tester code used for testing. > (developed by Daniel Hazelton -- modified slightly to now use 'take 6' > version for 'TinyLZO') > > Cheers, > Nitin > > On 5/28/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: >> (Using tes

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Nitin Gupta
On 5/28/07, Daniel Hazelton <[EMAIL PROTECTED]> wrote: On Monday 28 May 2007 10:40:31 Nitin Gupta wrote: > Hi, > > Attached is tester code used for testing. > (developed by Daniel Hazelton -- modified slightly to now use 'take 6' > version for 'TinyLZO') > > Cheers, > Nitin > I haven't tested w

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Daniel Hazelton
On Monday 28 May 2007 10:40:31 Nitin Gupta wrote: > Hi, > > Attached is tester code used for testing. > (developed by Daniel Hazelton -- modified slightly to now use 'take 6' > version for 'TinyLZO') > > Cheers, > Nitin > I haven't tested with version 6, but after removing the LZO_CHECK_MPOS_NON_

Re: [RFC] LZO de/compression support - take 6

2007-05-28 Thread Nitin Gupta
Hi, Attached is tester code used for testing. (developed by Daniel Hazelton -- modified slightly to now use 'take 6' version for 'TinyLZO') Cheers, Nitin On 5/28/07, Nitin Gupta <[EMAIL PROTECTED]> wrote: (Using tester program from Daniel) Following compares this kernel port ('take 6') vs ori