[
https://issues.apache.org/jira/browse/SPARK-19982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15930094#comment-15930094
]
Jose Soltren commented on SPARK-19982:
--------------------------------------
Pasting in the exception from the link in case the link dies:
Unable to generate an encoder for inner class
`test.org.apache.spark.sql.JavaDatasetSuite$SimpleJavaBean` without access to
the scope that this class was defined in. Try moving this class out of its
parent class.;
org.apache.spark.sql.AnalysisException: Unable to generate an encoder for inner
class `test.org.apache.spark.sql.JavaDatasetSuite$SimpleJavaBean` without
access to the scope that this class was defined in. Try moving this class out
of its parent class.;
at
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$$anonfun$2.applyOrElse(ExpressionEncoder.scala:264)
at
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$$anonfun$2.applyOrElse(ExpressionEncoder.scala:260)
at
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:243)
at
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$3.apply(TreeNode.scala:243)
at
org.apache.spark.sql.catalyst.trees.CurrentOrigin$.withOrigin(TreeNode.scala:53)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:242)
at
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformDown$1.apply(TreeNode.scala:248)
at
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$transformDown$1.apply(TreeNode.scala:248)
at
org.apache.spark.sql.catalyst.trees.TreeNode$$anonfun$4.apply(TreeNode.scala:265)
at scala.collection.Iterator$$anon$11.next(Iterator.scala:328)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at
scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:48)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:103)
at
scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:47)
at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:273)
at scala.collection.AbstractIterator.to(Iterator.scala:1157)
at
scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:265)
at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1157)
at
scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:252)
at scala.collection.AbstractIterator.toArray(Iterator.scala:1157)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformChildren(TreeNode.scala:305)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transformDown(TreeNode.scala:248)
at
org.apache.spark.sql.catalyst.trees.TreeNode.transform(TreeNode.scala:233)
at
org.apache.spark.sql.catalyst.encoders.ExpressionEncoder.resolve(ExpressionEncoder.scala:260)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:78)
at org.apache.spark.sql.Dataset.<init>(Dataset.scala:89)
at org.apache.spark.sql.SQLContext.createDataset(SQLContext.scala:507)
at org.apache.spark.sql.SQLContext.createDataset(SQLContext.scala:520)
at
test.org.apache.spark.sql.JavaDatasetSuite.testJavaBeanEncoder(JavaDatasetSuite.java:696)
> JavaDatasetSuite.testJavaBeanEncoder sometimes fails with "Unable to generate
> an encoder for inner class"
> ---------------------------------------------------------------------------------------------------------
>
> Key: SPARK-19982
> URL: https://issues.apache.org/jira/browse/SPARK-19982
> Project: Spark
> Issue Type: Bug
> Components: Tests
> Affects Versions: 2.1.0
> Reporter: Jose Soltren
> Labels: flaky-test
>
> JavaDatasetSuite.testJavaBeanEncoder fails sporadically with the error below:
> Unable to generate an encoder for inner class
> `test.org.apache.spark.sql.JavaDatasetSuite$SimpleJavaBean` without access to
> the scope that this class was defined in. Try moving this class out of its
> parent class.
> From https://spark-tests.appspot.com/test-logs/35475788
> [~vanzin] looked into this back in October and reported:
> I ran this test in a loop (both alone and with the rest of the spark-sql
> tests) and never got a failure. I even used the same JDK as Jenkins
> (1.7.0_51).
> Also looked at the code and nothing seems wrong. The errors is when an entry
> with the parent class name is missing from the map kept in OuterScopes.scala,
> but the test populates that map in its first line. So it doesn't look like a
> race nor some issue with weak references (the map uses weak values).
> public void testJavaBeanEncoder() {
> OuterScopes.addOuterScope(this);
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
