Yes, StatResGNT is unique among our Koine Greek modules for having quotes.
Koine NT manuscripts do not have quotes.
But the module has the wrong quotes for Modern Greek. They should be «double
chevron» for level 1 and “Double quotes” level 2.
That said, if the upstream had different quote marks for level 2, you’d have
never noticed visually when accents were stripped.
In Him,
DM
> On Mar 21, 2025, at 3:21 AM, David Haslam <dfh...@protonmail.com> wrote:
>
> Thanks DM,
>
> Very informative!
>
> Statistical Restoration Greek New Testament (StatResGNT) is an exception for
> a unique reason. Unlike any other Greek NT module, the project makes use of
> quotation marks for speech!
>
> These were never in any original Greek MSS, so it's a form of reverse
> engineering.
> cf. That's also the case with the added Pilcrow symbols ¶ to mark paragraphs.
>
> When a user selects to Hide Greek Accents in the front-end (e.g. Xiphos), the
> level 2 quotations lose their closing marks in the displayed text. This is
> what led me to revisit the topic.
>
> Aside: As a Windows user with BabelPad
> <https://www.babelstone.co.uk/Software/BabelPad.html> installed, I can enter
> any Unicode character into what I'm writing by looking it up in the Character
> Map tool, so I'd have no difficulty in inserting U+2019 into a search string.
> There are similar useful apps even for mobile platforms nowadays, such as
> UnicodePad Pro for iOS/iPadOS.
>
> Best regards,
>
> David
>
> Sent with Proton Mail <https://pr.tn/ref/SWXT9A5YZ67G> secure email.
>
> On Friday, March 21st, 2025 at 2:53 AM, DM Smith <dmsm...@crosswire.org>
> wrote:
>> I think I’ve finished my impact analysis on SWORD search code. I ran out of
>> time to do more. SWORD has several different search mechanisms. Some are
>> affected by the proposed change. Some are not.
>>
>> In all searches, the search request is normalized by the application by
>> calling UTF8GreekAccents (and a few others) to remove accents and stripText
>> is called to remove OSIS, GBF, ThML, TEI, …. markup and have simple markup
>> for added words, overline and xref notes. If the search is case insensitive,
>> it is converted to uppercase. The same is done for each piece of text that
>> is pulled out of the module.
>>
>> SEARCHTYPE_EXTERNAL - Lucene or Xapian
>> Both Lucene and Xapian, apply additional normalizations to their search
>> requests in the same fashion as the text. I didn’t examine Xapian source so
>> I’m not sure with Xapian. Lucene definitely does not have a problem. The
>> results are the same with or without U+2019 as it is not a token character.
>> I expect Xapian is similar.
>>
>> SEARCHTYPE_ENTRYATTR - entryAttrib (eg. Word//Lemma./G1234/) (Lemma with
>> dot means check components (Lemma.[1-9]) also)
>> I didn’t examine this. I’m not familiar with it. I’ll guess that it is no
>> worse than the next three and might not have an issue.
>>
>> SEARCHTYPE_PHRASE
>> The is a simple substring search.
>>
>> SEARCHTYPE_REGEX
>> The search request is compiled into a regular expression (there are 3
>> variants depending on what regex libraries are used to build SWORD lib.)
>> There is special code to allow a RE to match across 2 verses.
>>
>> SEARCHTYPE_MULTIWORD - multiword
>> The search request is split on spaces, ‘ ‘, (not whitespace) into needles
>> (as Troy calls them) and each needle is matched by simple substring in the
>> haystack.
>> This is an AND search. All needles need to be found for there to be a hit.
>> There is special code to allow a request to match across 2 verses.
>>
>> In these last 3 changing the UTF8GreekAccents to not remove U+2019 will
>> change the user experience for search.
>> They will have to include it if it is in the text. This is no different than
>> punctuation. which is not stripped from the text. This is no different from
>> U+0027 which might be used in the text for the same purpose.
>> They will have to either copy/paste the character or figure out how to enter
>> it. This is a non-obvious task. I outlined it in an earlier post.
>>
>> When it comes to accented/unaccented texts, I thought I’d look for how
>> unaccented texts handle the elision. Mark 2:17 is a good example of αλλα
>> being shortened to αλλ followed by οι.
>> I looked at all the Greek (grc) modules at CrossWire that have Mark 2:17.
>>
>> Here are the unaccented modules:
>> Antoniades Patriarchal Edition (1904/1912)
>> και ακουσας ο ιησους λεγει αυτοις ου χρειαν εχουσιν οι ισχυοντες ιατρου αλλ
>> οι κακως εχοντες ουκ ηλθον καλεσαι δικαιους αλλα αμαρτωλους εις μετανοιαν
>> The New Testament in the Original Greek: Byzantine Textform 2013
>> και ακουσας ο ιησους λεγει αυτοις ου χρειαν εχουσιν οι ισχυοντες ιατρου αλλ
>> οι κακως εχοντες ουκ ηλθον καλεσαι δικαιους αλλα αμαρτωλους εις μετανοιαν
>> Elzevir Textus Receptus (1624)
>> και ακουσας ο ιησους λεγει αυτοις ου χρειαν εχουσιν οι ισχυοντες ιατρου αλλ
>> οι κακως εχοντες ουκ ηλθον καλεσαι δικαιους αλλα αμαρτωλους εις μετανοιαν
>> Textus Receptus (1550/1894)
>> και ακουσας ο ιησους λεγει αυτοις ου χρειαν εχουσιν οι ισχυοντες ιατρου αλλ
>> οι κακως εχοντες ουκ ηλθον καλεσαι δικαιους αλλα αμαρτωλους εις μετανοιαν
>> Westcott and Hort with NA27/UBS4 variants
>> και ακουσας ο ιησους λεγει αυτοις [οτι] ου χρειαν εχουσιν οι ισχυοντες
>> ιατρου αλλ οι κακως εχοντες ουκ ηλθον καλεσαι δικαιους αλλα αμαρτωλους
>> Family 35
>> και ακουσας ο ιησους λεγει αυτοις ου χρειαν εχουσιν οι ισχυοντες ιατρου αλλ
>> οι κακως εχοντες ουκ ηλθον καλεσαι δικαιους αλλα αμαρτωλους εις μετανοιαν
>>
>> Here are the accented modules:
>> Apostolic Bible Polyglot Greek Text
>> και ακούσας ο Ιησούς λέγει αυτοίς ου χρείαν έχουσιν οι ισχύοντες ιατρού αλλ΄
>> οι κακώς έχοντες ουκ ήλθον καλέσαι δικαίους αλλά αμαρτωλούς εις μετάνοιαν
>> Morphologically Parsed Greek New Testament based on the SBLGNT
>> καὶ ἀκούσας ὁ Ἰησοῦς λέγει αὐτοῖς ⸀ὅτι Οὐ χρείαν ἔχουσιν οἱ ἰσχύοντες ἰατροῦ
>> ἀλλ’ οἱ κακῶς ἔχοντες· οὐκ ἦλθον καλέσαι δικαίους ἀλλὰ ⸀ἁμαρτωλούς.
>> Nestle GNT 1904
>> καὶ ἀκούσας ὁ Ἰησοῦς λέγει αὐτοῖς Οὐ χρείαν ἔχουσιν οἱ ἰσχύοντες ἰατροῦ ἀλλ’
>> οἱ κακῶς ἔχοντες· οὐκ ἦλθον καλέσαι δικαίους ἀλλὰ ἁμαρτωλούς.
>> Nestle-Aland, Novum Testamentum Graece, 28th Revised Edition
>> καὶ ἀκούσας ὁ Ἰησοῦς λέγει °αὐτοῖς °¹[ὅτι] οὐ χρείαν ἔχουσιν οἱ ἰσχύοντες
>> ἰατροῦ ἀλλ ᾿ οἱ κακῶς ἔχοντες· οὐκ ἦλθον καλέσαι δικαίους ἀλλ ᾿ ἁμαρτωλούς.
>> Nestle-Aland, Novum Testamentum Graece, 28th Revised Edition
>> καὶ ἀκούσας ὁ Ἰησοῦς λέγει °αὐτοῖς °¹[ὅτι] οὐ χρείαν ἔχουσιν οἱ ἰσχύοντες
>> ἰατροῦ ἀλλ ᾿ οἱ κακῶς ἔχοντες· οὐκ ἦλθον καλέσαι δικαίους ἀλλ ᾿ ἁμαρτωλούς.
>> The Greek New Testament: SBL Edition
>> καὶ ἀκούσας ὁ Ἰησοῦς λέγει αὐτοῖς ⸀ὅτι Οὐ χρείαν ἔχουσιν οἱ ἰσχύοντες ἰατροῦ
>> ἀλλʼ οἱ κακῶς ἔχοντες· οὐκ ἦλθον καλέσαι δικαίους ἀλλὰ ⸀ἁμαρτωλούς.
>> Statistical Restoration Greek New Testament
>> Καὶ ἀκούσας, ὁ ˚Ἰησοῦς λέγει αὐτοῖς, “Οὐ χρείαν ἔχουσιν οἱ ἰσχύοντες ἰατροῦ,
>> ἀλλʼ οἱ κακῶς ἔχοντες. Οὐκ ἦλθον καλέσαι δικαίους, ἀλλὰ ἁμαρτωλούς.”
>> Tregelles' Greek New Testament
>> καὶ ἀκούσας ὁ Ἰησοῦς λέγει αὐτοῖς, Οὐ χρείαν ἔχουσιν οἱ ἰσχύοντες ἰατροῦ,
>> ἀλλ᾽ οἱ κακῶς ἔχοντες. οὐκ ἦλθον καλέσαι δικαίους, ἀλλὰ ἁμαρτωλούς.
>> Tischendorf's 8th edition GNT
>> καὶ ἀκούσας ὁ Ἰησοῦς λέγει αὐτοῖς· οὐ χρείαν ἔχουσιν οἱ ἰσχύοντες
>> ἰατροῦ ἀλλ’ οἱ κακῶς ἔχοντες· οὐκ ἦλθον καλέσαι δικαίους ἀλλὰ ἁμαρτωλούς.
>>
>> Note that all the unaccented Greek texts do not have αλλ΄ οι.
>>
>> My recommendation is that we make no changes:
>> 1) If we take the CrossWire unaccented Koine Greek modules as representative
>> of the proper way to handle U+2019, then it should be stripped along with
>> accents.
>> 2) It changes current behavior that has not been an issue previously.
>> 3) It is difficult for an end user to enter U+2019 and to know that it is a
>> particular 1 of several glyphs that look the same.
>> 4) Ancient texts, e.g. 4th-5th century, did not have accents or the elision
>> mark.
>>
>> In Him,
>> DM
>>
>>> On Mar 19, 2025, at 7:16 PM, DM Smith <dmsm...@crosswire.org> wrote:
>>>
>>> David,
>>>
>>> I’ve examined the use of Lucene in SWORD and it treats ’ U+2019 as a
>>> non-token character. SWORD uses the standard analyzer which is predicated
>>> on modern European languages with a nod to CJK. The ASCII apostrophe is
>>> looked at and in some circumstances it is a token character, but most often
>>> is ignored.
>>>
>>> So from a Lucene perspective, the change does not affect it and would be
>>> safe.
>>>
>>> While in the code, I found that Xapian is the preferred external search
>>> mechanism for indexed searches. From a quick look at Xapian and SWORD’s use
>>> of it, it is a far better search engine and has language awareness. I
>>> haven’t checked to see how it handles this elision in its tokenizer.
>>>
>>> There are several other search mechanisms in SWORD that I’m checking out.
>>>
>>> Aside, on my Mac, it is easy to input U+2019 from a USA keyboard input, but
>>> requires knowing how to do it and what it is. When I turned on Polytonic
>>> Greek as my keyboard input, it wasn’t possible. Apparently elision is
>>> rarely used in modern Greek. I had thought if anyone knew Koine (ancient)
>>> Greek and could readily type it, that they’d know how to input it. I’m not
>>> so sure.
>>>
>>> Since you like your chat AI agents, ask it/them what the first and second
>>> level quote characters are for Greek. It surprised me. It’s not U+2019 or
>>> U+0027 for either.
>>>
>>> DM
>>>
>>>
>>>> On Mar 18, 2025, at 12:55 PM, DM Smith <dmsm...@crosswire.org> wrote:
>>>>
>>>> Apparently we are talking past each other.
>>>>
>>>> I understand the argument. The elision mark is not an accent. It is a
>>>> letter character when used between letters but a punctuation character (so
>>>> a word break character) when at the beginning or end of words. It has been
>>>> requested of the Unicode consortium to make it a letter character when it
>>>> follows a Greek letter, but this has not been done. The goal of the
>>>> request is that double clicking on the Greek word ending with U+2019 would
>>>> also select the apostrophe in the same way that it does when it is in the
>>>> middle of a character sequence.
>>>>
>>>> I’m saying if the filter does double duty for both presentation and
>>>> normalization of search requests, then both have to work well.
>>>>
>>>> That U+2019 is visually similar to U+0027 and an end user will use the
>>>> keyboard to type U+0027 when U+2019 is required. This will not work as the
>>>> exact code point has to match.
>>>>
>>>> Over the years, I’ve found that simple code changes often break other
>>>> parts of the code.
>>>>
>>>> I’ve requested to examine the search code in Xiphos to see if another
>>>> filter is applied that strips U+2019. If so, then your request probably
>>>> will work. However, if Xiphos needs to be changed in addition to SWORD lib
>>>> to accommodate the change, then probably every frontend would need to be
>>>> changed.
>>>>
>>>> Also, I need to dig into the Lucene index creation and Lucene search to
>>>> make sure it doesn’t require indexes to be rebuilt.
>>>>
>>>> DM
>>>>
>>>>> On Mar 18, 2025, at 3:21 AM, David Haslam <dfh...@protonmail.com> wrote:
>>>>>
>>>>> We don't hide U+2019 in any other context.
>>>>> It's use in the KJV for possessives does not hinder search!
>>>>>
>>>>> Why the obsession with search, when it has no bearing on my semantic
>>>>> argument?
>>>>> An elision mark is simply not an accent!!! Why is this so hard to
>>>>> understand?
>>>>>
>>>>> We wouldn't hide U+2019 in French if we saw it used frequently as an
>>>>> elision mark!!!
>>>>> Albeit most French modules to date simply use U+0027 (unlike our KJV).
>>>>>
>>>>> Best regards,
>>>>>
>>>>> David
>>>>>
>>>>> Sent with Proton Mail <https://pr.tn/ref/SWXT9A5YZ67G> secure email.
>>>>>
>>>>> On Tuesday, March 18th, 2025 at 1:40 AM, DM Smith <dmsm...@crosswire.org>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>> On Mar 17, 2025, at 5:24 PM, David Haslam <dfh...@protonmail.com> wrote:
>>>>>>>
>>>>>>> My argument is simple & straightforward.
>>>>>>
>>>>>> Your argument is that display and search should have nothing to do with
>>>>>> each other.
>>>>>>
>>>>>>>
>>>>>>> When you hide diacritics, you ought not to be hiding punctuation marks.
>>>>>>>
>>>>>>> Why is this so contentious?
>>>>>>
>>>>>> Software changes can have unintended consequences. It needs to be
>>>>>> carefully considered.
>>>>>>
>>>>>>>
>>>>>>> In what world does how a module displays require that punctuation be
>>>>>>> hidden?
>>>>>>
>>>>>> In a world where the filter is also used for search.
>>>>>>>
>>>>>>> A quotation mark is not an accent!
>>>>>>>
>>>>>>> David
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 17, 2025 at 21:06, DM Smith <dmsm...@crosswire.org
>>>>>>> <mailto:On Mon, Mar 17, 2025 at 21:06, DM Smith <<a href=>> wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On Mar 17, 2025, at 4:01 PM, Karl Kleinpaste <k...@kleinpaste.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> That is, if search success depends on the specificity of whether
>>>>>>>>> accents or points are enabled, you're probably doing something wrong.
>>>>>>>>
>>>>>>>> Absolutely.
>>>>>>>>
>>>>>>>> What we are discussing is whether an apostrophe should be stripped out
>>>>>>>> or not. The apostrophe on the keyboard is U+0027. The apostrophe in
>>>>>>>> the Greek is recommended to be U+2019. They look nearly the same.
>>>>>>>> There are a few other Unicode apostrophes that have the same
>>>>>>>> appearance.
>>>>>>>>
>>>>>>>> If the apostrophe isn’t stripped out, then the user input and the text
>>>>>>>> have to agree on which of the several it is. There is no way for the
>>>>>>>> end user to know which. I end up copying from the text to make sure I
>>>>>>>> have the right one.
>>>>>>>>
>>>>>>>> I believe the same filter that turns on and off the display of accents
>>>>>>>> is used for searching. David is suggesting that the filter is taking
>>>>>>>> out legitimate level 2 quotation marks from the display when it
>>>>>>>> shouldn’t. I’m suggesting that it needs to remove them for the sake of
>>>>>>>> the search.
>>>>>>>>
>>>>>>>> DM
>>>>>>> _______________________________________________
>>>>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>>>>> Instructions to unsubscribe/change your settings at above page
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>>> Instructions to unsubscribe/change your settings at above page
>>>>
>>>> _______________________________________________
>>>> sword-devel mailing list: sword-devel@crosswire.org
>>>> http://crosswire.org/mailman/listinfo/sword-devel
>>>> Instructions to unsubscribe/change your settings at above page
>>>
>>> _______________________________________________
>>> sword-devel mailing list: sword-devel@crosswire.org
>>> http://crosswire.org/mailman/listinfo/sword-devel
>>> Instructions to unsubscribe/change your settings at above page
>>
>
> _______________________________________________
> sword-devel mailing list: sword-devel@crosswire.org
> http://crosswire.org/mailman/listinfo/sword-devel
> Instructions to unsubscribe/change your settings at above page
_______________________________________________
sword-devel mailing list: sword-devel@crosswire.org
http://crosswire.org/mailman/listinfo/sword-devel
Instructions to unsubscribe/change your settings at above page
