This is an automated email from the ASF dual-hosted git repository.
krickert pushed a change to branch OPENNLP-1850-3-dl
in repository https://gitbox.apache.org/repos/asf/opennlp.git
omit b6dc2418f OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
omit 062322e22 OPENNLP-1850 Fail loud on corrupt document-classification
model output
omit 7e1c4e55d OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
omit f3dc9b992 OPENNLP-1850 Harden fail-loud paths in the DL components
omit 284afa576 OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
omit 702d392c8 OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
omit aff3fd44f OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
omit e945009ea OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
omit b6cd17380 OPENNLP-1850 Per-language NormalizationProfile registry (2c)
omit 57e2b5833 OPENNLP-1850 Layered Term model: Term, TermAnalyzer (2b)
omit a450069ef OPENNLP-1850 UAX #29 word tokenizer: WordSegmenter,
WordTokenizer, WordType (2a)
omit 08de0d35c OPENNLP-1850 Offset/alignment layer: Alignment, AlignedText,
buildAligned, *Aligned (1b)
add 1a6b27387 OPENNLP-1850 Fail loud on a structurally-malformed
confusables line (engine)
add 9af6d92bd OPENNLP-1850 Offset/alignment layer: Alignment, AlignedText,
buildAligned, *Aligned (1b)
add e51758922 OPENNLP-1850 UAX #29 word tokenizer: WordSegmenter,
WordTokenizer, WordType (2a)
add dc02b9e85 OPENNLP-1850 Fail loud on a Word_Break line missing its ';'
(tokenizer)
add 82cb041e8 OPENNLP-1850 Layered Term model: Term, TermAnalyzer (2b)
add be8d26e31 OPENNLP-1850 Per-language NormalizationProfile registry (2c)
add a345f48b2 OPENNLP-1850 Resolve Norwegian nb/nn to the Norwegian
profile (profiles)
add 3c6acd4e1 OPENNLP-1850 Offset-safe, Unicode-aware input normalization
in the DL components
add 85d45f13f OPENNLP-1850 Add OffsetMappingNameFinder capability
interface and a findInOriginal end-to-end test
add 8c206cadc OPENNLP-1850 Resolve overlapping chunk spans and compose the
input alignment
add 46cac9839 OPENNLP-1850 Add real-model chunk-boundary eval tests; drop
dead label constants
add 264b20ac8 OPENNLP-1850 Harden fail-loud paths in the DL components
add fd9479f18 OPENNLP-1850 Fail fast on null finder input; fix the GPU
eval test options
add 4dc78ffad OPENNLP-1850 Fail loud on corrupt document-classification
model output
add 10d43cf7b OPENNLP-1850 Fully-qualify TokenNameFinder javadoc links in
NameFinderDL
add be98bd611 OPENNLP-1850 Reject non-finite logits in softmax, not just
NaN (dl)
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (b6dc2418f)
\
N -- N -- N refs/heads/OPENNLP-1850-3-dl (be98bd611)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
No new revisions were added by this update.
Summary of changes:
.../opennlp/dl/doccat/DocumentCategorizerDL.java | 8 +-
.../dl/doccat/DocumentCategorizerDLTest.java | 13 +++
.../tools/tokenize/uax29/WordBreakProperty.java | 9 +-
.../opennlp/tools/util/normalizer/Confusables.java | 98 ++++++++++++----------
.../util/normalizer/NormalizationProfiles.java | 4 +-
.../tokenize/uax29/WordBreakPropertyTest.java | 16 ++++
.../tools/util/normalizer/ConfusablesLoadTest.java | 53 ++++++++++++
.../util/normalizer/NormalizationProfilesTest.java | 19 ++++-
8 files changed, 169 insertions(+), 51 deletions(-)
create mode 100644
opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/util/normalizer/ConfusablesLoadTest.java
