[
https://issues.apache.org/jira/browse/TIKA-4754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18086504#comment-18086504
]
Hudson commented on TIKA-4754:
------------------------------
SUCCESS: Integrated in Jenkins build Tika ยป tika-main-jdk17 #1409 (See
[https://ci-builds.apache.org/job/Tika/job/tika-main-jdk17/1409/])
TIKA-4754 -- move to bloom filters for common_tokens (#2876) (github:
[https://github.com/apache/tika/commit/de9433e3cf0e080fe4dd899a3d09daa9b31eef55])
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/min
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/vol
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/smn
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ace
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/rue
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tur
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ydd
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/olo
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ron
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ckb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kab
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tet
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/oss
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lus
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/amh
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/nno
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/vls
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/csb
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/vol
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/yor
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/olo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/szy
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/glv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/smo
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ron
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/sna
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hyw
* (edit)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/tokens/TikaEvalTokenizer.java
* (edit) tika-eval/tika-eval-core/pom.xml
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tay
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/aze
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ewe
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ben
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/nqo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/cor
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ina
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/pan
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/cor
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/eus
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kaa
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ava
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ukr
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/smn
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kpv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/vep
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ron
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/wln
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tso
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/cor
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/uzb
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/fao
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/est
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/khm
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/nob
* (delete)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/textstats/CommonTokensHellinger.java
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/vls
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/jav
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tel
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/fas
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/sme
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mkd
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/spa
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/eng
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ava
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ewe
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/zho
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tam
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/vie
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mal
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/be-x-old
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/grn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lit
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/vro
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ido
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/uig
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kaa
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/fry
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kur
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kaz
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mya
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/cym
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/trv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/swe
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/div
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/snd
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kir
* (edit)
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/tools/CommonTokenOverlapCounter.java
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kir
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/chv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/sna
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/dan
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tha
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/msa
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ban
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ces
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/jav
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ext
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/guj
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/zul
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/fas
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ami
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/urd
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tum
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/pol
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/pus
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tgl
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/isl
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ukr
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/avk
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hye
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bxr
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/sat
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ibo
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/jbo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/jbo
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lug
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/nep
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/amh
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hin
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/sme
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/fao
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bod
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ksh
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/che
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tgl
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ind
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ace
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/diq
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/nso
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hun
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mhr
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/zho
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mon
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/sun
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bak
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kaa
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/orm
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/smo
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/azb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/nep
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/fra
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/war
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/oss
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/som
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/slv
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/gla
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ckb
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hin
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/cnh
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/pap
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ydd
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/som
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/vep
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/afr
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tha
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/xmf
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ben
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ckb
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kan
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ind
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/srp
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bar
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/xmf
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/min
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kir
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mlg
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hau
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ban
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/gom
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kur
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ita
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/szl
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tam
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/por
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/san
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/aka
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mwl
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tir
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ami
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tsn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/che
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kan
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hun
* (delete)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/textstats/CommonTokensKLDivergence.java
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/fin
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mzn
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/fry
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mwl
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/slv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/xho
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mlg
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/roh
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/sin
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/avk
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/div
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/nld
* (delete)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/textstats/CommonTokensBhattacharyya.java
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bel
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lat
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hsb
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/aze
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/cnh
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/myv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ind
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/dsb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/isl
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kin
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lez
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/rue
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/nqo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/chv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/heb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/fra
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mrj
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/san
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/som
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tur
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/sme
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/alt
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ukr
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kat
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/rus
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/war
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/dag
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hak-x-rom
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/vro
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ydd
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/dag
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/yue
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/fin
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bod
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/wln
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/pol
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mkd
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hil
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/deu
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mlg
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tat
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ksh
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tgk
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/nya
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/eng
* (add)
tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hak-x-rom
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bul
* (edit)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/tokens/CommonTokenCountManager.java
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hsb
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lim
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/sgs
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/arg
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bar
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hun
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ext
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hak-x-rom
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/urd
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/diq
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/skr
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/udm
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bre
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ile
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ibo
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lim
* (edit)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/tokens/LangModel.java
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/jpn
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/olo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/uig
* (delete)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/textstats/CommonTokensKLDNormed.java
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mwl
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tum
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mya
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bcl
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tgk
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/orm
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/slk
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/est
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mlt
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/frr
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/che
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/skr
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/srp
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/aka
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mhr
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tuk
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lat
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tel
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mrj
* (edit)
tika-eval/tika-eval-core/src/test/java/org/apache/tika/eval/core/langid/LangIdTest.java
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bar
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mzn
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tat
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/pnb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/zul
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lav
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/pus
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ile
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/sun
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/chv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ace
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lao
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/pfl
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tyv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kat
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kpv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hin
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/pnb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/yor
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/gom
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/pam
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kha
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bjn
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/deu
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/pam
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ara
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ksh
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/nno
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/vol
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/gsw
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kor
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/fra
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/sqi
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bul
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hye
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hil
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ces
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bre
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/gla
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/diq
* (add)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/tokens/CommonTokensBloom.java
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lim
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ewe
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/snd
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/sqi
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mlt
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ltz
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/cat
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/afr
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kat
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kpv
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/yue
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kor
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/udm
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bjn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bel
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ina
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/amh
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bod
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ori
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hsb
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/pap
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/epo
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/stq
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/alt
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/uig
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/nds
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kha
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/epo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/nno
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/gla
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/uzb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tay
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/nld
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tir
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/pfl
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/cdo-x-rom
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/jpn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/rus
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/szl
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hrv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tir
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/vie
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ltz
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/dan
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tgk
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ilo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bul
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/be-x-old
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/slk
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/xho
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/dan
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/pus
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ita
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hau
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tuk
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tat
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mya
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kur
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/gle
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mal
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/rue
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/sah
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ina
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/dag
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/cat
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lao
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lug
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tum
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lez
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/cym
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hrv
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/cnh
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ilo
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ori
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tet
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/cos
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kan
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/myv
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/msa
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/deu
* (edit)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/textstats/CommonTokens.java
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/jpn
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/yue
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kab
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lez
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bxr
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/nob
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mar
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/eng
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hrv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/arg
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/srp
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tur
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mar
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bre
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lav
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kor
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lus
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/vep
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lfn
* (delete)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/textstats/CommonTokensCosine.java
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bcl
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/sah
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/smn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ilo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/frr
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/eus
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/khm
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/trv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/nya
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/war
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ile
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/swe
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/heb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/guj
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mon
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ext
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/bxr
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/csb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/frr
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lit
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/arg
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/guj
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/be-x-old
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/stq
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/vro
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/nds
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/pfl
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/sna
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/por
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/snd
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ido
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/aze
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/azb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/sah
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/epo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mzn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/swh
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/slv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tuk
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/nld
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/fin
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hyw
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bcl
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/gsw
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hye
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/nso
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/gle
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/pan
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tsn
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mlt
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/gle
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/est
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/spa
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/pam
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/sin
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/glv
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/avk
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/skr
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/fas
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/zho
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/szy
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ell
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/udm
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/azb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/khm
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/spa
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/cos
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/eus
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ell
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/glg
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mkd
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/gom
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kaz
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tyv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mon
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/cos
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tet
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/wln
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kaz
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/fry
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/vls
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ara
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tay
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/trv
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tgl
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ceb
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bjn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/nqo
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/hyw
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/oss
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/sat
* (add)
tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/cdo-x-rom
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/aka
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/csb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/dsb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/tsn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ceb
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/szy
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/gsw
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/myv
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/hil
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ltz
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ami
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/jav
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/afr
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/hau
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/nep
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mhr
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/min
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/rus
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ibo
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/sun
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/vie
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ori
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/roh
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/div
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bak
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/nya
* (add)
tika-eval/tika-eval-core/src/test/java/org/apache/tika/eval/core/tokens/tools/CommonTokensBloomGenerator.java
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/msa
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lug
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/cdo-x-rom
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ava
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lfn
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ben
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/lit
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kin
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/pnb
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lao
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/grn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lat
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/kin
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/san
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/ceb
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ita
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/sin
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tam
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/glv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/asm
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/heb
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/bel
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/smo
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/dsb
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/asm
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/grn
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/mrj
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/pap
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tyv
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/xho
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/glg
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/xmf
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/kha
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/nds
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ara
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/sgs
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/cat
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/jbo
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/kab
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/lus
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/pan
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ban
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/glg
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/sqi
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ces
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/uzb
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/orm
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/nso
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/mal
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lav
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/swh
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/nob
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/swh
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/stq
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/sat
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/swe
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tel
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/zul
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/tso
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/szl
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/isl
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/slk
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/cym
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/pol
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/roh
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/mar
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/lfn
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/urd
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tso
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/por
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/yor
* (add) tika-eval/tika-eval-core/src/test/resources/common_tokens/alt
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/ido
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/ell
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/fao
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/asm
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/sgs
* (delete) tika-eval/tika-eval-core/src/main/resources/common_tokens/tha
* (add) tika-eval/tika-eval-core/src/main/resources/common_tokens_bloom/bak
> Switch to bloom filters for common tokens in tika-eval
> ------------------------------------------------------
>
> Key: TIKA-4754
> URL: https://issues.apache.org/jira/browse/TIKA-4754
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Trivial
>
> This can bring the tika-eval jar from 22mb -> 8.5mb without much of a change
> in stats. We could go lower, but then there's more of a diff because of
> expected bloom filter limitations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
