This is an automated email from the ASF dual-hosted git repository.
rzo1 pushed a change to branch OPENNLP-1220
in repository https://gitbox.apache.org/repos/asf/opennlp.git
from d056dd27 OPENNLP-1220 - Add support for Byte Pair Encoding (BPE)
add 159e7865 Checkstyle
No new revisions were added by this update.
Summary of changes:
.../main/java/opennlp/tools/tokenize/BPEModel.java | 49 +++++++------
.../java/opennlp/tools/tokenize/BPETokenizer.java | 14 ++--
.../tools/tokenize/BPETokenizerFactory.java | 83 +++++++++++++++-------
.../tools/tokenize/BPETokenizerTrainer.java | 73 ++++++++++++-------
4 files changed, 142 insertions(+), 77 deletions(-)
