xushiyan commented on code in PR #13829:
URL: https://github.com/apache/hudi/pull/13829#discussion_r2345777583
##########
docker/hoodie/hadoop/spark_base/Dockerfile:
##########
Review Comment:
python 3.8 too old. let's go to 3.13 if everything works. if not, at least
go with 3.12
##########
packaging/hudi-hive-sync-bundle/pom.xml:
##########
@@ -92,6 +92,7 @@
<include>com.esotericsoftware:kryo-shaded</include>
<include>com.esotericsoftware:minlog</include>
<include>org.objenesis:objenesis</include>
+
<include>com.github.ben-manes.caffeine:caffeine</include>
Review Comment:
what is the reason of this fix? this has to be separate out as another pr to
highlight what it fixes. this affects release artifacts
##########
hudi-sync/hudi-hive-sync/run_sync_tool.sh:
##########
@@ -46,7 +46,7 @@ HIVE_JDBC=`ls ${HIVE_HOME}/lib/hive-jdbc-*.jar | tr '\n' ':'`
if [ -z "${HIVE_JDBC}" ]; then
HIVE_JDBC=`ls ${HIVE_HOME}/lib/hive-jdbc-*.jar | grep -v handler | tr '\n'
':'`
fi
-HIVE_JARS=$HIVE_METASTORE:$HIVE_SERVICE:$HIVE_EXEC:$HIVE_JDBC
+HIVE_JARS=$HIVE_METASTORE:$HIVE_SERVICE:$HIVE_EXEC:$HIVE_JDBC:${HIVE_HOME}/lib/calcite-core-1.16.0.jar:${HIVE_HOME}/lib/libfb303-0.9.3.jar
Review Comment:
raise a separate pr to highlight dep fix
##########
docker/hoodie/hadoop/base/Dockerfile:
##########
@@ -22,7 +22,7 @@ USER root
# Default to UTF-8 file.encoding
ENV LANG C.UTF-8
-ARG HADOOP_VERSION=2.8.4
+ARG HADOOP_VERSION=3.3.4
Review Comment:
we should be able to simplify this without maintaining a base hadoop image.
use official one https://hub.docker.com/r/apache/hadoop/tags
do this in compose.yml. use platform let arm machine enter compatibility
mode:
```yaml
hdfs-namenode:
image: apache/hadoop:3.4
platform: linux/amd64
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
