Fast Search could benefit from this, but SWORD uses plain text as an input to
Lucene index creation and searching. The implementation of the Lucene analyzer
that SWORD uses for all texts uses ASCII space as a word break. The presence of
various zero width characters would not help. Nor would using <w>…</w><w>…</w>
without an ASCII space.
Plain text, also called strip text, does double duty. Presentation without
markup and preparation for indexing. For latinate texts, this works fine. I
don’t know if LocalStripFilter could help in this.
In Him,
DM
> On May 1, 2025, at 7:46 AM, Peter von Kaehne <ref...@gmx.net> wrote:
>
> I think this is not difficult per se, but it should be properly encoded.
>
> <w> seems correct, using zero with characters seems not correct.
>
> Peter
>
> Sent from Outlook for iOS <https://aka.ms/o0ukef>
> From: sword-devel <sword-devel-boun...@crosswire.org> on behalf of David
> Haslam <dfh...@protonmail.com>
> Sent: Thursday, May 1, 2025 11:30 am
> To: sword-devel mailing list <sword-devel@crosswire.org>
> Cc: David Haslam <df.has...@btinternet.com>
> Subject: [sword-devel] Proposal for a new SWORD filter to display word
> dividers
>
> I wish to propose that we design in a new SWORD filter.
>
> The conf key would be:
> GlobalOptionFilter=ShowWordDividers
>
> In the writing systems for the various languages of SE Asia (Thai, Khmer,
> Lao, Myanmar) there is [generally] no space between words.
>
> In this respect, they are like many European languages before the start of
> silent reading
> <https://www.amazon.com/Space-Between-Words-Origins-Medieval/dp/080474016X>.
> The descriptive term is Scriptura Continua.
>
> Some Bible translations for this region are already making use of one of the
> ZERO WIDTH characters to invisibly mark the divisions between lexical words.
> Options include:
> U+200B ZERO WIDTH SPACE
> U+200C ZERO WIDTH NON-JOINER
> U+FEFF ZERO WIDTH NO BREAK SPACE
> They exclude:
> U+200D ZERO WIDTH JOINER
> A further possibility, even without requiring a full study Bible with
> Strong's, etc, is to simply wrap each lexical word within the OSIS w element.
> One without any OSIS attributes would suffice for this purpose. Likewise, for
> the seg element.
>
> My proposal is that we design a feature to show/hide word dividers by
> displaying them using a suitable visible but non-intrusive character.
> My suggestion is to use this Unicode character by default:
>
> U+00B7 MIDDLE DOT
>
> We could even allow the actual visible character to be specified in a second
> conf key, thus:
>
> VisibleWordDivider=U+00B7
>
> Benefits would include:
> Helps with language learning to know where lexical words start and end
> Helps with front-end search for whole words, exact phrase or all words
> Helps with checking the accuracy of Bible translations by clearly displaying
> lexical word boundaries at the touch of a single key in the front-end
> Paves the way for Study Bible with the addition of Strong's mark-up, etc.
>
> Here's a sample of Khmer verse text with the MIDDLE DOT as the visible word
> divider:
> Obad.1.1
> នេះ·ជា·សុបិន·និមិត្ដ·របស់·លោក·អូបាឌា
> ព្រះអម្ចាស់·ជា·ព្រះ·មាន·បន្ទូល·ពី·ក្រុង·អេដំម ។
> យើង·បាន·ឮ·ដំណឹង·មក·ពី·ព្រះអម្ចាស់ គឺ·មាន·ទូត·ម្នាក់·បាន·បញ្ជូន·ឲ្យ·ទៅ
> ក្នុង·ចំណោម·ជន·ជាតិ·ទាំង·ឡាយ·ដោយ·ពាក្យ·ថា "ចូរ·ក្រោក·ឡើង !
> ចូរ·យើង·ក្រោក·ឡើង·ធ្វើ·ចម្បាំង·ទាស់·និង·គេ"
>
> cf. Here's what it looks like with the ZWSP as the invisible word divider:
> Obad.1.1
> នេះជាសុបិននិមិត្ដរបស់លោកអូបាឌា
> ព្រះអម្ចាស់ជាព្រះមានបន្ទូលពីក្រុងអេដំម ។
> យើងបានឮដំណឹងមកពីព្រះអម្ចាស់ គឺមានទូតម្នាក់បានបញ្ជូនឲ្យទៅ
> ក្នុងចំណោមជនជាតិទាំងឡាយដោយពាក្យថា "ចូរក្រោកឡើង !
> ចូរយើងក្រោកឡើងធ្វើចម្បាំងទាស់និងគេ"
>
> If SWORD developers agree that my proposal merits consideration, please would
> you start on the software development.
>
>
> Best regards,
>
> David
>
> Sent with Proton Mail <https://pr.tn/ref/SWXT9A5YZ67G> secure email.
>
> _______________________________________________
> sword-devel mailing list: sword-devel@crosswire.org
> http://crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
_______________________________________________
sword-devel mailing list: sword-devel@crosswire.org
http://crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
