[
https://issues.apache.org/jira/browse/PIG-5318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nandor Kollar updated PIG-5318:
-------------------------------
Description:
There are several failing cases when executing the unit tests with Spark 2.2:
{code}
org.apache.pig.test.TestAssert#testNegativeWithoutFetch
org.apache.pig.test.TestAssert#testNegative
org.apache.pig.test.TestEvalPipeline2#testNonStandardDataWithoutFetch
org.apache.pig.test.TestScalarAliases#testScalarErrMultipleRowsInInput
org.apache.pig.test.TestStore#testCleanupOnFailureMultiStore
org.apache.pig.test.TestStoreInstances#testBackendStoreCommunication
org.apache.pig.test.TestStoreLocal#testCleanupOnFailureMultiStore
{code}
All of these are related to fixes/changes in Spark.
TestAssert, TestScalarAliases and TestEvalPipeline2 failures could be fixed by
asserting on the message of the exception's root cause, looks like on Spark 2.2
the exception is wrapped into an additional layer.
TestStore and TestStoreLocal failure are also a test related problems: looks
like SPARK-7953 is fixed in Spark 2.2
The root cause of TestStoreInstances is yet to be found out.
was:
There are sever failing cases when executing the unit tests with Spark 2.2:
{code}
org.apache.pig.test.TestAssert#testNegativeWithoutFetch
org.apache.pig.test.TestAssert#testNegative
org.apache.pig.test.TestEvalPipeline2#testNonStandardDataWithoutFetch
org.apache.pig.test.TestScalarAliases#testScalarErrMultipleRowsInInput
org.apache.pig.test.TestStore#testCleanupOnFailureMultiStore
org.apache.pig.test.TestStoreInstances#testBackendStoreCommunication
org.apache.pig.test.TestStoreLocal#testCleanupOnFailureMultiStore
{code}
All of these are related to fixes/changes in Spark.
TestAssert, TestScalarAliases and TestEvalPipeline2 failures could be fixed by
asserting on the message of the exception's root cause, looks like on Spark 2.2
the exception is wrapped into an additional layer.
TestStore and TestStoreLocal failure are also a test related problems: looks
like SPARK-7953 is fixed in Spark 2.2
The root cause of TestStoreInstances is yet to be found out.
> Unit test failures on Pig on Spark with Spark 2.2
> -------------------------------------------------
>
> Key: PIG-5318
> URL: https://issues.apache.org/jira/browse/PIG-5318
> Project: Pig
> Issue Type: Bug
> Components: spark
> Reporter: Nandor Kollar
> Assignee: Nandor Kollar
> Attachments: PIG-5318_1.patch
>
>
> There are several failing cases when executing the unit tests with Spark 2.2:
> {code}
> org.apache.pig.test.TestAssert#testNegativeWithoutFetch
> org.apache.pig.test.TestAssert#testNegative
> org.apache.pig.test.TestEvalPipeline2#testNonStandardDataWithoutFetch
> org.apache.pig.test.TestScalarAliases#testScalarErrMultipleRowsInInput
> org.apache.pig.test.TestStore#testCleanupOnFailureMultiStore
> org.apache.pig.test.TestStoreInstances#testBackendStoreCommunication
> org.apache.pig.test.TestStoreLocal#testCleanupOnFailureMultiStore
> {code}
> All of these are related to fixes/changes in Spark.
> TestAssert, TestScalarAliases and TestEvalPipeline2 failures could be fixed
> by asserting on the message of the exception's root cause, looks like on
> Spark 2.2 the exception is wrapped into an additional layer.
> TestStore and TestStoreLocal failure are also a test related problems: looks
> like SPARK-7953 is fixed in Spark 2.2
> The root cause of TestStoreInstances is yet to be found out.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
