[
https://issues.apache.org/jira/browse/SPARK-19982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928660#comment-15928660
]
Jose Soltren commented on SPARK-19982:
--------------------------------------
[~cloud_fan] added this test as some work related to SPARK-11954. Wenchen, do
you have any thoughts as to why this might be failing intermittently?
Likely not, but I wonder if this has anything to do with outerScopes being a
lazy val in object OuterScopes. Then, possibly, in very rare instances,
Analyzer.scala:ResolveNewInstance could hit the (outer == null) branch and
throw this exception.
We run all the Spark unit tests about a dozen times a night and this has failed
on average twice a month since last May (which is as far back as my data goes).
> JavaDatasetSuite.testJavaBeanEncoder sometimes fails with "Unable to generate
> an encoder for inner class"
> ---------------------------------------------------------------------------------------------------------
>
> Key: SPARK-19982
> URL: https://issues.apache.org/jira/browse/SPARK-19982
> Project: Spark
> Issue Type: Bug
> Components: Tests
> Affects Versions: 2.1.0
> Reporter: Jose Soltren
>
> JavaDatasetSuite.testJavaBeanEncoder fails sporadically with the error below:
> Unable to generate an encoder for inner class
> `test.org.apache.spark.sql.JavaDatasetSuite$SimpleJavaBean` without access to
> the scope that this class was defined in. Try moving this class out of its
> parent class.
> From https://spark-tests.appspot.com/test-logs/35475788
> [~vanzin] looked into this back in October and reported:
> I ran this test in a loop (both alone and with the rest of the spark-sql
> tests) and never got a failure. I even used the same JDK as Jenkins
> (1.7.0_51).
> Also looked at the code and nothing seems wrong. The errors is when an entry
> with the parent class name is missing from the map kept in OuterScopes.scala,
> but the test populates that map in its first line. So it doesn't look like a
> race nor some issue with weak references (the map uses weak values).
> public void testJavaBeanEncoder() {
> OuterScopes.addOuterScope(this);
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
