This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch OPENNLP-1850-4-docs
in repository https://gitbox.apache.org/repos/asf/opennlp.git
omit 0bb8b2da2 OPENNLP-1850 Docs review nits: populate ids2Labels example;
rule-based hyphenation
omit 663710ca1 OPENNLP-1850 Mirror the Extended_Pictographic emoji caveat
in the tokenizer manual
omit f24edeb0c OPENNLP-1850 Tighten normalizer manual wording (review nits)
omit ec9fc7634 OPENNLP-1850 Document the supplementary-dash offset shift in
the DL fold options
omit 7ab13e543 OPENNLP-1850 Document the offset-aware substitution folds
(quotes, digits, ellipsis, bullets, umlaut)
omit b6960500f OPENNLP-1850 Name the OffsetMappingNameFinder capability
interface in the manual
omit 25c027f81 OPENNLP-1850 Document the offset-aware normalization
pipeline (buildAligned)
omit 9bf85e75b OPENNLP-1850 Document Unicode normalization, the UAX #29
tokenizer, and DL handling
omit 5a1114c0d OPENNLP-1850 Make mergeOverlappingSpans O(n log n) (dl)
omit d4fec508f OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
omit e10be7d9b OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
omit f7f26d376 OPENNLP-1850 Fail loud on corrupt document-classification
model output
omit 330325910 OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
omit ba6c5dbc1 OPENNLP-1850 Harden fail-loud paths in the DL components
omit 61e0100a2 OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
omit 004073926 OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
omit 6c9a8f192 OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
omit 349399083 OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
omit 93618ce40 OPENNLP-1850 Resolve Norwegian nb/nn to the Norwegian
profile (profiles)
omit 9cac6484e OPENNLP-1850 Per-language NormalizationProfile registry (2c)
omit e35e8591d OPENNLP-1850 Layered Term model: Term, TermAnalyzer (2b)
omit dd1906d0e OPENNLP-1850 Route WordBreakProperty/ExtendedPictographic
through the shared Lazy holder
omit a46b2f3bb OPENNLP-1850 Fail loud on a Word_Break line missing its ';'
(tokenizer)
omit aa3522eb9 OPENNLP-1850 UAX #29 word tokenizer: WordSegmenter,
WordTokenizer, WordType (2a)
omit 9dc7d5106 OPENNLP-1850 Offset/alignment layer: Alignment, AlignedText,
buildAligned, *Aligned (1b)
omit 69f513616 OPENNLP-1850 Dedup expanding folds via CharClass.substitute;
share a Lazy holder (engine)
add 20226fbbe OPENNLP-1850 Dedup the expanding folds via
CharClass.substitute (engine)
add b24c9ee3d OPENNLP-1850 Offset/alignment layer: Alignment, AlignedText,
buildAligned, *Aligned (1b)
add 47480171c OPENNLP-1850 UAX #29 word tokenizer: WordSegmenter,
WordTokenizer, WordType (2a)
add 3fae8aad6 OPENNLP-1850 Fail loud on a Word_Break line missing its ';'
(tokenizer)
add 55dbeb4b2 OPENNLP-1850 Layered Term model: Term, TermAnalyzer (2b)
add f600dfd55 OPENNLP-1850 Per-language NormalizationProfile registry (2c)
add 70ec7a3df OPENNLP-1850 Resolve Norwegian nb/nn to the Norwegian
profile (profiles)
add da0d84fd0 OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
add cb99cd4e5 OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
add dd9b495a3 OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
add 808535466 OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
add 553b89acf OPENNLP-1850 Harden fail-loud paths in the DL components
add d7d3f2813 OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
add 22e9314ee OPENNLP-1850 Fail loud on corrupt document-classification
model output
add 2e4e04175 OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
add 0bd4f058f OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
add 57ceefdb2 OPENNLP-1850 Make mergeOverlappingSpans O(n log n) (dl)
add 7640014db OPENNLP-1850 Document Unicode normalization, the UAX #29
tokenizer, and DL handling
add 0c3b7f01d OPENNLP-1850 Document the offset-aware normalization
pipeline (buildAligned)
add d514e0a74 OPENNLP-1850 Name the OffsetMappingNameFinder capability
interface in the manual
add 8bb30116c OPENNLP-1850 Document the offset-aware substitution folds
(quotes, digits, ellipsis, bullets, umlaut)
add f2d6d0e30 OPENNLP-1850 Document the supplementary-dash offset shift in
the DL fold options
add 783dc4408 OPENNLP-1850 Tighten normalizer manual wording (review nits)
add e5b4d887f OPENNLP-1850 Mirror the Extended_Pictographic emoji caveat
in the tokenizer manual
add c0fb2ebcb OPENNLP-1850 Docs review nits: populate ids2Labels example;
rule-based hyphenation
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (0bb8b2da2)
\
N -- N -- N refs/heads/OPENNLP-1850-4-docs (c0fb2ebcb)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../tools/tokenize/uax29/ExtendedPictographic.java | 28 +++++++---
.../tools/tokenize/uax29/WordBreakProperty.java | 30 +++++++---
.../src/main/java/opennlp/tools/util/Lazy.java | 65 ----------------------
.../opennlp/tools/util/normalizer/Confusables.java | 31 ++++++++---
4 files changed, 66 insertions(+), 88 deletions(-)
delete mode 100644
opennlp-core/opennlp-runtime/src/main/java/opennlp/tools/util/Lazy.java
