[
https://issues.apache.org/jira/browse/LUCENE-6978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113020#comment-15113020
]
Steve Rowe commented on LUCENE-6978:
------------------------------------
My Jenkins found a reproducible locale-triggered DIH failure on branch_5x:
{noformat}
[junit4] Suite:
org.apache.solr.handler.dataimport.TestVariableResolverEndToEnd
[...]
[junit4] 2> 24391 T75 C8 oasc.SolrException.log ERROR Exception while
processing: FIRST document : SolrInputDocument(fields:
[select_keyword_s=SELECT,
id=1]):org.apache.solr.handler.dataimport.DataImportHandlerException: Malformed
/ non-existent locale: nn_NO Processing Document # 1
[...]
[junit4] 2> 24392 T75 C8
oasup.LogUpdateProcessorFactory$LogUpdateProcessor.finish [collection1]
webapp=null path=null
params={synchronous=true&command=full-import&wt=xml&indent=true&clean=true&dataConfig=<dataConfig>+%0a<dataSource+name%3D"hsqldb"+driver%3D"${dataimporter.request.dots.in.hsqldb.driver}"+url%3D"jdbc:hsqldb:mem:."+/>+%0a<document+name%3D"TestEvaluators">+%0a<entity+name%3D"FIRST"+processor%3D"SqlEntityProcessor"+dataSource%3D"hsqldb"++query%3D"select++1+as+id,++'SELECT'+as+SELECT_KEYWORD,++CURRENT_TIMESTAMP+as+FIRST_TS+from+DUAL+"+>%0a++<field+column%3D"SELECT_KEYWORD"+name%3D"select_keyword_s"+/>+%0a++<entity+name%3D"SECOND"+processor%3D"SqlEntityProcessor"+dataSource%3D"hsqldb"+transformer%3D"TemplateTransformer"++++query%3D"${dataimporter.functions.encodeUrl(FIRST.SELECT_KEYWORD)}++1+as+SORT,++CURRENT_TIMESTAMP+as+SECOND_TS,++'${dataimporter.functions.formatDate(FIRST.FIRST_TS,+'yyyy',+'nn_NO')}'+as+SECOND1_S,+++'PORK'+AS+MEAT,++'GRILL'+AS+METHOD,++'ROUND'+AS+CUTS,++'BEEF_CUTS'+AS+WHATKIND+from+DUAL+WHERE+1%3D${FIRST.ID}+UNION+${dataimporter.functions.encodeUrl(FIRST.SELECT_KEYWORD)}++2+as+SORT,++CURRENT_TIMESTAMP+as+SECOND_TS,++'${dataimporter.functions.formatDate(FIRST.FIRST_TS,+'yyyy',+'nn_NO')}'+as+SECOND1_S,+++'FISH'+AS+MEAT,++'FRY'+AS+METHOD,++'SIRLOIN'+AS+CUTS,++'BEEF_CUTS'+AS+WHATKIND+from+DUAL+WHERE+1%3D${FIRST.ID}+ORDER+BY+SORT+">%0a+++<field+column%3D"SECOND_S"+name%3D"second_s"+/>+%0a+++<field+column%3D"SECOND1_S"+name%3D"second1_s"+/>+%0a+++<field+column%3D"second2_s"+template%3D"${dataimporter.functions.formatDate(SECOND.SECOND_TS,+'yyyy',+'nn_NO')}"+/>+%0a+++<field+column%3D"second3_s"+template%3D"${dih.functions.formatDate(SECOND.SECOND_TS,+'yyyy',+'nn_NO')}"+/>+%0a+++<field+column%3D"METHOD"+name%3D"${SECOND.MEAT}_s"/>%0a+++<field+column%3D"CUTS"+name%3D"${SECOND.WHATKIND}_mult_s"/>%0a++</entity>%0a</entity>%0a</document>+%0a</dataConfig>+%0a&commit=true}{deleteByQuery=*:*}
0 9
[junit4] 2> 24393 T75 C8 oasc.SolrException.log ERROR Full Import
failed:java.lang.RuntimeException: java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException: Malformed /
non-existent locale: nn_NO Processing Document # 1
[junit4] 2> at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:271)
[junit4] 2> at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:417)
[junit4] 2> at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:481)
[junit4] 2> at
org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:200)
[junit4] 2> at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
[junit4] 2> at
org.apache.solr.core.SolrCore.execute(SolrCore.java:2083)
[junit4] 2> at
org.apache.solr.util.TestHarness.query(TestHarness.java:311)
[junit4] 2> at
org.apache.solr.handler.dataimport.TestVariableResolverEndToEnd.test(TestVariableResolverEndToEnd.java:40)
[...]
[junit4] 2> Caused by: java.lang.RuntimeException:
org.apache.solr.handler.dataimport.DataImportHandlerException: Malformed /
non-existent locale: nn_NO Processing Document # 1
[junit4] 2> at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:417)
[junit4] 2> at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:330)
[junit4] 2> at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:233)
[junit4] 2> ... 46 more
[junit4] 2> Caused by:
org.apache.solr.handler.dataimport.DataImportHandlerException: Malformed /
non-existent locale: nn_NO Processing Document # 1
[junit4] 2> at
org.apache.solr.handler.dataimport.DateFormatEvaluator.evaluate(DateFormatEvaluator.java:100)
[junit4] 2> at
org.apache.solr.handler.dataimport.VariableResolver.resolveEvaluator(VariableResolver.java:136)
[junit4] 2> at
org.apache.solr.handler.dataimport.VariableResolver.resolve(VariableResolver.java:100)
[junit4] 2> at
org.apache.solr.handler.dataimport.VariableResolver.replaceTokens(VariableResolver.java:155)
[junit4] 2> at
org.apache.solr.handler.dataimport.ContextImpl.replaceTokens(ContextImpl.java:257)
[junit4] 2> at
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)
[junit4] 2> at
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:244)
[junit4] 2> at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:476)
[junit4] 2> at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:515)
[junit4] 2> at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:415)
[junit4] 2> ... 48 more
[junit4] 2> Caused by: java.util.IllformedLocaleException: Invalid
subtag: nn_NO [at index 0]
[junit4] 2> at
java.util.Locale$Builder.setLanguageTag(Locale.java:2311)
[junit4] 2> at
org.apache.solr.handler.dataimport.DateFormatEvaluator.evaluate(DateFormatEvaluator.java:98)
[junit4] 2> ... 57 more
[junit4] 2>
[...]
[junit4] 2> NOTE: reproduce with: ant test
-Dtestcase=TestVariableResolverEndToEnd -Dtests.method=test
-Dtests.seed=E764DCBE41663305 -Dtests.slow=true
-Dtests.linedocsfile=/home/jenkins/lucene-data/enwiki.random.lines.txt
-Dtests.locale=nn-NO -Dtests.timezone=America/Mendoza -Dtests.asserts=true
-Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.04s J2 | TestVariableResolverEndToEnd.test <<<
[junit4] > Throwable #1: junit.framework.AssertionFailedError
[junit4] > at
__randomizedtesting.SeedInfo.seed([E764DCBE41663305:6F30E364EF9A5EFD]:0)
[junit4] > at junit.framework.Assert.fail(Assert.java:48)
[junit4] > at junit.framework.Assert.assertTrue(Assert.java:20)
[junit4] > at junit.framework.Assert.assertTrue(Assert.java:27)
[junit4] > at
org.apache.solr.handler.dataimport.TestVariableResolverEndToEnd.test(TestVariableResolverEndToEnd.java:47)
[junit4] > at java.lang.Thread.run(Thread.java:745)
[...]
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene54): {},
docValues:{}, sim=DefaultSimilarity, locale=nn-NO, timezone=America/Mendoza
[junit4] 2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation
1.7.0_79 (64-bit)/cpus=16,threads=1,free=493820680,total=514326528
[junit4] 2> NOTE: All tests run in this JVM: [TestVariableResolver,
TestSimplePropertiesWriter, TestURLDataSource, TestDocBuilder2,
TestSqlEntityProcessorDelta, TestDateFormatTransformer, TestRegexTransformer,
TestClobTransformer, TestDataConfig, TestNestedChildren,
TestXPathEntityProcessor, TestFileListWithLineEntityProcessor,
TestVariableResolverEndToEnd]
[junit4] Completed [25/38 (1!)] on J2 in 0.60s, 1 test, 1 failure <<<
FAILURES!
{noformat}
> Make LuceneTestCase use language tags instead of parsing locales by hand
> ------------------------------------------------------------------------
>
> Key: LUCENE-6978
> URL: https://issues.apache.org/jira/browse/LUCENE-6978
> Project: Lucene - Core
> Issue Type: Improvement
> Components: modules/test-framework
> Reporter: Uwe Schindler
> Assignee: Uwe Schindler
> Fix For: 5.5, Trunk
>
> Attachments: LUCENE-6978-5x.patch, LUCENE-6978.patch,
> LUCENE-6978.patch, LUCENE-6978.patch
>
>
> Since we are on Java 7, the JDK supports standardized language tags as
> identifiers for Locales. Previous versions of JDK were missing a constructor
> from {{Locale#toString()}} back to a locale, so we had our own, which was
> broken several times after the JDK changed their Locale internals.
> This patch will do the following:
> - When printing the reproduce line, it will use {{Locale#getLanguageTag()}},
> so you can identify the locale in standardized form. Most notable change is
> (next to more flexibility around asian languages) the change away from
> undescores. So it prints "en-US", not "en_US".
> - The code that parses a locale uses Locale's Builder and sets the language
> tag. This will fail if the tag is invalid! A trap is
> {{Locale#forLanguageTag}}, because this one silently returns root locale if
> unparseable...
> - The random locale is choosen from all language tags, which are extracted
> from the JDK as a {{String[]}} array.
> I would also like to place {{Locale#forLanguageTag}} on the forbidden list
> and disallow directly calling {{Locale#toString()}}, the latter is legacy API
> (according to Java 7 Javadocs). This would fail code that calls toString()
> directly, e.g. when formatting stuff like {{"my Locale: " + locale}}. Of
> course we cannot catch all bad uses.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
