[
https://issues.apache.org/jira/browse/HIVE-30011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18110942#comment-18110942
]
shml commented on HIVE-30011:
-----------------------------
Hi, I have a patch ready for this and will open a PR shortly. Could a committer
add me to the contributors role so I can assign it to myself? Thanks.
> MR jobs submitted outside ExecDriver (MergeFileTask, ColumnTruncateTask,
> MRCompactor) miss the JDK 17 --add-opens flags
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: HIVE-30011
> URL: https://issues.apache.org/jira/browse/HIVE-30011
> Project: Hive
> Issue Type: Bug
> Components: Query Processor, Transactions
> Affects Versions: 4.1.0
> Reporter: shml
> Priority: Major
>
> HIVE-28869 makes Hive append the JDK 17 {{--add-opens}} flags to the JVMs it
> launches itself: Tez AM / containers ({{DagUtils}}, {{TezSessionState}}), MR
> AM / tasks ({{ExecDriver}}) and the WebHCat controller
> ({{TempletonControllerJob}}). The MR side of that injection lives in
> {{ExecDriver.execute()}}, but three code paths submit MR jobs through
> {{JobClient}} directly and never go through {{ExecDriver}}:
> || Path || Trigger ||
> | {{ql/io/merge/MergeFileTask}} | {{ALTER TABLE ... CONCATENATE}} with
> {{hive.execution.engine=mr}} |
> | {{ql/io/rcfile/truncate/ColumnTruncateTask}} | {{TRUNCATE TABLE ... COLUMNS
> (...)}} on RCFile tables |
> | {{ql/txn/compactor/MRCompactor}} | ACID compaction launched by the
> metastore {{Worker}}; still the default for full-CRUD tables because
> {{hive.compactor.crud.query.based}} defaults to {{false}} |
> None of them touches {{mapreduce.map.java.opts}},
> {{mapreduce.reduce.java.opts}} or {{yarn.app.mapreduce.am.command-opts}}, so
> on a JDK 17 cluster whose {{mapred-site.xml}} does not carry the flags the
> task JVMs fail on the first reflective access into {{java.base}}.
> *Reproduction* (JDK 17 on both the submitter and the containers, no
> {{--add-opens}} in {{mapred-site.xml}}):
> {code:sql}
> SET hive.execution.engine=mr;
> CREATE TABLE t_orc (id INT, s STRING) STORED AS ORC;
> INSERT INTO t_orc VALUES (1,'a');
> INSERT INTO t_orc VALUES (2,'b');
> ALTER TABLE t_orc CONCATENATE;
> {code}
> Task attempt log:
> {noformat}
> Error: java.lang.reflect.InaccessibleObjectException: Unable to make field
> private volatile java.lang.String java.net.URI.string accessible:
> module java.base does not "opens java.net" to unnamed module
> at org.apache.hadoop.hive.common.StringInternUtils.<clinit>
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init
> {noformat}
> An {{INSERT ... SELECT}} on the same cluster succeeds because {{ExecDriver}}
> injects the flags.
> *Why CI does not catch it*: the MiniMR / qtest drivers run tasks in the local
> job runner inside the test JVM, where the {{*.java.opts}} keys are ignored
> and the surefire {{argLine}} already opens the packages. Only a real YARN
> submission exercises the container command line.
> *Proposed fix*: move the injection into a shared helper and call it from
> every MR submission path.
> {code:java}
> // org.apache.hadoop.hive.common.JavaVersionUtils
> public static void addOpensFlags(Configuration job) {
> String addOpens = getAddOpensFlags();
> for (String key : new String[] {"mapreduce.map.java.opts",
> "mapreduce.reduce.java.opts", "yarn.app.mapreduce.am.command-opts"}) {
> String current = job.get(key);
> job.set(key, (current == null ? "" : current) + addOpens);
> }
> }
> {code}
> * {{ExecDriver}} calls the helper instead of its private {{addOpensFlags}} /
> {{addJavaOpts}}.
> * {{MergeFileTask.execute}}, {{ColumnTruncateTask.execute}} and
> {{MRCompactor.launchCompactionJob}} call it right before {{new
> JobClient(job)}}.
> * Append semantics are kept, so cluster / user {{java.opts}} such as heap
> sizes survive. String-literal keys keep {{hive-common}} free of a
> {{hadoop-mapreduce-client-core}} dependency, as {{ExecDriver}} already does
> today.
> * Unit test in {{TestJavaVersionUtils}}: an existing {{-Xmx}} value is
> preserved and all three keys end with the flag set.
> Not in scope: {{MapredLocalTask}} (map-join local task) inherits
> {{HADOOP_CLIENT_OPTS}} from the parent process, which
> {{bin/ext/hiveserver2.sh}} already exports since HIVE-26473.
> *Verification*: an equivalent patch on a 3.1.x-based build was run on a
> 3-node YARN cluster (Hadoop 3.4.3, JDK 17.0.20 on submitter and containers,
> cluster-side {{--add-opens}} removed). Before the patch {{ALTER TABLE ...
> CONCATENATE}} failed as above; after it the AM and task
> {{launch_container.sh}} of the merge job, the RCFile column-truncate job and
> the metastore-launched major compaction all carry the flag set and the jobs
> succeed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
