msokolov commented on a change in pull request #722: LUCENE-8863: handle some
edge cases in Kuromoji DictionaryBuilder, en…
URL: https://github.com/apache/lucene-solr/pull/722#discussion_r295505621
##########
File path:
lucene/analysis/kuromoji/src/java/org/apache/lucene/analysis/ja/dict/BinaryDictionary.java
##########
@@ -46,13 +50,31 @@
public static final String POSDICT_HEADER = "kuromoji_dict_pos";
public static final int VERSION = 1;
+ static final String CLASSPATH = "classpath:";
+ static final String FILE = "file:";
+
+ private final String resourcePrefix;
private final ByteBuffer buffer;
private final int[] targetMapOffsets, targetMap;
private final String[] posDict;
private final String[] inflTypeDict;
private final String[] inflFormDict;
protected BinaryDictionary() throws IOException {
+ this(CLASSPATH);
+ }
+
+ /**
+ * @param resourcePrefix - where to load resources (dictionaries) from.
Either "file:", followed
+ * by a filesystem directory and optionally a filename-prefix, or
"classpath:", followed by a
+ * classpath location prefix, or just plain "classpath:" to use this class's
name as the prefix.
+ */
+ protected BinaryDictionary(String resourcePrefix) throws IOException {
Review comment:
enum is a good idea; I'll update the patch. I added "file:" to enable unit
testing. We can't load a test dictionary from the classpath, I think. Another
possibility would be to pass in dictionaries directly somehow. EG we could
maybe have {{...DictionaryBuilder}} {{FST.save()}} to an in-memory buffer, but
this seemed like a more intrusive change since we can already save to a set of
files.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
