I think I caused confusion by saying it is an experimental compression driver but I guess I mixed that up with the other compression forms we have in the engine but do not really support.
So, me bad here.
Peter
Sent from my mobile. Please forgive shortness, typos and weird autocorrects.
-------- Original Message --------
Subject: Re: [sword-devel] Segfault in LZSS code
From: "Troy A. Griffitts"
To: David Haslam
CC:
Maybe I don't understand. It is the default because it is the first compression type we supported and was the compression type of STEP data files (which previously was an open standard for Bible data; not having anything to do with the STEP Bible Project).
The problem would be the same if ZIP was the default and you gave the ZIP compression driver LZSS data files.
In summary, if you specify the wrong driver for the data files of your module, you will get undefined behaviour. We could probably do a better job adding sanity checks to be sure the data we retrieve from a module is likely valid values, but these checks would be at the lowest level of engine use and be called very frequently, resulting is slower results all to handle a misconfiguration, so I'm not rushing to add these checks.
The bottom line is, give the LZSS drivers LZSS data and give the Zip drivers Zip data, etc.
Happy Sunday!
TroyOn February 28, 2021 1:45:12 AM MST, David Haslam <dfh...@protonmail.com> wrote:Still unanswered is Bastian’s goodquestion “Why is it the default then?” DavidSent from ProtonMail MobileOn Sun, Feb 28, 2021 at 03:06, Troy A. Griffitts <scr...@crosswire.org> wrote:While I agree we probably shouldn't segfault, this is a bit like
renaming xyz.exe to xyz.doc and trying to open it in Word.
Not that is matters, but the details are that the LZSS decompressor
treats the entire block as uncompressed binary (because it doesn't
encounter any compression block signatures, because the data is not LZSS
data but instead ZIP encoded data) and does nothing with any of it,
returning the zip compressed data as-is, resulting in binary gibberish
when we reference it, so when we pull offset and size values from the
"uncompressed" block (which really isn't uncompressed), we get
essentially random numbers... which causes things to crash. Not sure
what Word does when you give it a ".doc" file which is a binary, or
MySQL does when you cat /dev/random > /var/data/mysql/ibdata1, but I
would bet it doesn't behave very well.
I would say the solution to this is don't give the LZSS filters ZIP
compressed data. I can try to put some sanity checks in when I read the
offset, size values from the "uncompressed" data and might be able to
prevent a crash.
Troy
On 2/27/21 10:19 AM, Bastian Germann wrote:
> Am 27.02.21 um 17:47 schrieb ref...@gmx.net:
>> Could you put a bug report into JIRA?
>
> https://tracker.crosswire.org/browse/API-247
>
>> As such the LZSS code is experimental and should not be relied upon.
>>
>
> Why is it the default then? (according to
> https://wiki.crosswire.org/DevTools:conf_Files#Required_Elements_with_defaults)
> _______________________________________________
> sword-devel mailing list: sword-devel@crosswire.org
> http://crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
_______________________________________________
sword-devel mailing list: sword-devel@crosswire.org
http://crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
_______________________________________________ sword-devel mailing list: sword-devel@crosswire.org http://crosswire.org/mailman/listinfo/sword-devel Instructions to unsubscribe/change your settings at above page
