[
https://issues.apache.org/jira/browse/FLINK-37337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Feng Jin updated FLINK-37337:
-----------------------------
Description:
During testing of Flink in Kubernetes Application mode, an issue was
encountered when submitting SQL scripts using the {*}SQL Client Embedded
Mode{*}.
The SQL Client JAR is included in the {{--jar}} parameter by default. This
parameter is converted to {{user.artifacts.artifact-list . And we will get the
following exception. }}{{}}
{code:java}
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException:
/home/feng/flink-cluster/image-build/flink/opt/flink-sql-client-2.0-SNAPSHOT.jar
(No such file or directory) │
│ at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown
Source) ~[?:?]
│
│ at
java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown
Source) ~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown
Source) ~[?:?]
│
│ at
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown
Source) ~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
~[?:?]
│
│ at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
~[?:?]
│
│ at
org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifacts(ArtifactFetchManager.java:126)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.fetchArtifacts(KubernetesApplicationClusterEntrypoint.java:159)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT] │
│ ... 8 more
│
│ Caused by: java.io.FileNotFoundException:
/home/feng/flink-cluster/image-build/flink/opt/flink-sql-client-2.0-SNAPSHOT.jar
(No such file or directory)
│
│ at java.base/java.io.FileInputStream.open0(Native Method) ~[?:?]
│
│ at java.base/java.io.FileInputStream.open(Unknown Source) ~[?:?]
│
│ at java.base/java.io.FileInputStream.<init>(Unknown Source) ~[?:?]
│
│ at
org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:52)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:133)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.client.program.artifact.FsArtifactFetcher.fetch(FsArtifactFetcher.java:45)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifact(ArtifactFetchManager.java:152)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:73)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown
Source) ~[?:?]
│
│ at
java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown
Source) ~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown
Source) ~[?:?]
│
│ at
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown
Source) ~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
~[?:?]
│
│ at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
~[?:?]
│
│ at
org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifacts(ArtifactFetchManager.java:126)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.fetchArtifacts(KubernetesApplicationClusterEntrypoint.java:159)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT] │
│ ... 8 more{code}
In sql-client.sh, there is a logic that uses --jar to include the sql-client
JAR if it is not in the classpath.
{code:java}
# check if SQL client is already in classpath and must not be shipped manually
if [[ "$CC_CLASSPATH" =~ .*flink-sql-client.*.jar ]]; then # start client
without jar
exec "$JAVA_RUN" $FLINK_ENV_JAVA_OPTS $JVM_ARGS "${log_setting[@]}"
-classpath "`manglePathList "$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
org.apache.flink.table.client.SqlClient "$@"# check if SQL client jar is in /opt
elif [ -n "$FLINK_SQL_CLIENT_JAR" ]; then # start client with jar
exec "$JAVA_RUN" $FLINK_ENV_JAVA_OPTS $JVM_ARGS "${log_setting[@]}"
-classpath "`manglePathList
"$CC_CLASSPATH:$FLINK_SQL_CLIENT_JAR:$INTERNAL_HADOOP_CLASSPATHS"`"
org.apache.flink.table.client.SqlClient "$@" --jar "`manglePath
$FLINK_SQL_CLIENT_JAR`"{code}
was:
During testing of Flink in Kubernetes Application mode, an issue was
encountered when submitting SQL scripts using the {*}SQL Client Embedded
Mode{*}.
The SQL Client JAR is included in the {{--jar}} parameter by default. This
parameter is converted to {{user.artifacts.artifact-list . And we will get the
following exception. }}{{}}
{code:java}
Caused by: java.lang.RuntimeException: java.io.FileNotFoundException:
/home/feng/flink-cluster/image-build/flink/opt/flink-sql-client-2.0-SNAPSHOT.jar
(No such file or directory) │
│ at org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown
Source) ~[?:?]
│
│ at
java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown
Source) ~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown
Source) ~[?:?]
│
│ at
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown
Source) ~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
~[?:?]
│
│ at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
~[?:?]
│
│ at
org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifacts(ArtifactFetchManager.java:126)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.fetchArtifacts(KubernetesApplicationClusterEntrypoint.java:159)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT] │
│ ... 8 more
│
│ Caused by: java.io.FileNotFoundException:
/home/feng/flink-cluster/image-build/flink/opt/flink-sql-client-2.0-SNAPSHOT.jar
(No such file or directory)
│
│ at java.base/java.io.FileInputStream.open0(Native Method) ~[?:?]
│
│ at java.base/java.io.FileInputStream.open(Unknown Source) ~[?:?]
│
│ at java.base/java.io.FileInputStream.<init>(Unknown Source) ~[?:?]
│
│ at
org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:52)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:133)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.client.program.artifact.FsArtifactFetcher.fetch(FsArtifactFetcher.java:45)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifact(ArtifactFetchManager.java:152)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:73)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown
Source) ~[?:?]
│
│ at
java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown
Source) ~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown
Source) ~[?:?]
│
│ at
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown
Source) ~[?:?]
│
│ at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
~[?:?]
│
│ at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
~[?:?]
│
│ at
org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifacts(ArtifactFetchManager.java:126)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
│
│ at
org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.fetchArtifacts(KubernetesApplicationClusterEntrypoint.java:159)
~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT] │
│ ... 8 more{code}
{{ }}
> Deploy script SQL Client in Embedded Mode to Kubernetes Results in
> FileNotFoundException
> ----------------------------------------------------------------------------------------
>
> Key: FLINK-37337
> URL: https://issues.apache.org/jira/browse/FLINK-37337
> Project: Flink
> Issue Type: Bug
> Components: Table SQL / Client
> Reporter: Feng Jin
> Priority: Major
>
> During testing of Flink in Kubernetes Application mode, an issue was
> encountered when submitting SQL scripts using the {*}SQL Client Embedded
> Mode{*}.
> The SQL Client JAR is included in the {{--jar}} parameter by default. This
> parameter is converted to {{user.artifacts.artifact-list . And we will get
> the following exception. }}{{}}
> {code:java}
> Caused by: java.lang.RuntimeException: java.io.FileNotFoundException:
> /home/feng/flink-cluster/image-build/flink/opt/flink-sql-client-2.0-SNAPSHOT.jar
> (No such file or directory) │
> │ at
> org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:321)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at
> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:75)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown
> Source) ~[?:?]
> │
> │ at
> java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown
> Source) ~[?:?]
> │
> │ at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
> ~[?:?]
> │
> │ at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown
> Source) ~[?:?]
> │
> │ at
> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown
> Source) ~[?:?]
> │
> │ at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
> ~[?:?]
> │
> │ at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
> ~[?:?]
> │
> │ at
> org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifacts(ArtifactFetchManager.java:126)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at
> org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.fetchArtifacts(KubernetesApplicationClusterEntrypoint.java:159)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT] │
> │ ... 8 more
>
> │
> │ Caused by: java.io.FileNotFoundException:
> /home/feng/flink-cluster/image-build/flink/opt/flink-sql-client-2.0-SNAPSHOT.jar
> (No such file or directory)
> │
> │ at java.base/java.io.FileInputStream.open0(Native Method) ~[?:?]
>
> │
> │ at java.base/java.io.FileInputStream.open(Unknown Source) ~[?:?]
>
> │
> │ at java.base/java.io.FileInputStream.<init>(Unknown Source) ~[?:?]
>
> │
> │ at
> org.apache.flink.core.fs.local.LocalDataInputStream.<init>(LocalDataInputStream.java:52)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at
> org.apache.flink.core.fs.local.LocalFileSystem.open(LocalFileSystem.java:133)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at
> org.apache.flink.client.program.artifact.FsArtifactFetcher.fetch(FsArtifactFetcher.java:45)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at
> org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifact(ArtifactFetchManager.java:152)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at
> org.apache.flink.util.function.FunctionUtils.lambda$uncheckedFunction$2(FunctionUtils.java:73)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown
> Source) ~[?:?]
> │
> │ at
> java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(Unknown
> Source) ~[?:?]
> │
> │ at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
> ~[?:?]
> │
> │ at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown
> Source) ~[?:?]
> │
> │ at
> java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown
> Source) ~[?:?]
> │
> │ at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
> ~[?:?]
> │
> │ at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
> ~[?:?]
> │
> │ at
> org.apache.flink.client.program.artifact.ArtifactFetchManager.fetchArtifacts(ArtifactFetchManager.java:126)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT]
> │
> │ at
> org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint.fetchArtifacts(KubernetesApplicationClusterEntrypoint.java:159)
> ~[flink-dist-2.0-SNAPSHOT.jar:2.0-SNAPSHOT] │
> │ ... 8 more{code}
> In sql-client.sh, there is a logic that uses --jar to include the sql-client
> JAR if it is not in the classpath.
> {code:java}
> # check if SQL client is already in classpath and must not be shipped manually
> if [[ "$CC_CLASSPATH" =~ .*flink-sql-client.*.jar ]]; then # start client
> without jar
> exec "$JAVA_RUN" $FLINK_ENV_JAVA_OPTS $JVM_ARGS "${log_setting[@]}"
> -classpath "`manglePathList "$CC_CLASSPATH:$INTERNAL_HADOOP_CLASSPATHS"`"
> org.apache.flink.table.client.SqlClient "$@"# check if SQL client jar is in
> /opt
> elif [ -n "$FLINK_SQL_CLIENT_JAR" ]; then # start client with jar
> exec "$JAVA_RUN" $FLINK_ENV_JAVA_OPTS $JVM_ARGS "${log_setting[@]}"
> -classpath "`manglePathList
> "$CC_CLASSPATH:$FLINK_SQL_CLIENT_JAR:$INTERNAL_HADOOP_CLASSPATHS"`"
> org.apache.flink.table.client.SqlClient "$@" --jar "`manglePath
> $FLINK_SQL_CLIENT_JAR`"{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
