Koji Noguchi created PIG-5439:
---------------------------------
Summary: Support Spark 3 and drop SparkShim
Key: PIG-5439
URL: https://issues.apache.org/jira/browse/PIG-5439
Project: Pig
Issue Type: Improvement
Components: spark
Reporter: Koji Noguchi
Assignee: Koji Noguchi
Support Pig-on-Spark to run on spark3.
Initial version would only run up to Spark 3.2.4 and not on 3.3 or 3.4.
This is due to log4j mismatch.
After moving to log4j2 (PIG-5426), we can move Spark to 3.3 or higher.
So far, not all unit/e2e tests pass with the proposed patch but at least
compilation goes through.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
