alexeykudinkin commented on issue #6278: URL: https://github.com/apache/hudi/issues/6278#issuecomment-1207104704
Can you please re-run your command re-ordering following properties: in your original command it was ``` --class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer \ --conf spark.shuffle.service.enabled=true \ --conf spark.default.parallelism=500 \ --conf spark.dynamicAllocation.enabled=true \ --conf spark.dynamicAllocation.initialExecutors=3 \ --conf spark.dynamicAllocation.cachedExecutorIdleTimeout=90s \ --conf spark.serializer=org.apache.spark.serializer.KryoSerializer \ --conf spark.app.name=SPKNG_TST_SCTN_ITM \ ``` But instead, please first specify all Spark properties only after that specify --class ...HoodieDeltaStreamer like following: ``` --conf spark.shuffle.service.enabled=true \ --conf spark.default.parallelism=500 \ --conf spark.dynamicAllocation.enabled=true \ --conf spark.dynamicAllocation.initialExecutors=3 \ --conf spark.dynamicAllocation.cachedExecutorIdleTimeout=90s \ --conf spark.serializer=org.apache.spark.serializer.KryoSerializer \ --conf spark.app.name=SPKNG_TST_SCTN_ITM \ --class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer \ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
