Hi Dan, the notation of "flink-jobmanager/10.98.253.58:8081" is not a problem. It is how java.net.InetAddress stringifies a resolved address (with both hostname and IP).
How did you configure the SQL client to work with a Kubernetes Session? Afaik this is not a documented, tested and officially supported feature (this doesn't mean we should not support it -- apparently it is something we should do rather soon ;) ). Best, Robert On Fri, Sep 11, 2020 at 5:25 AM Dan Hill <quietgol...@gmail.com> wrote: > I just tried using the Flink SQL Client. A simple job is not running > because it cannot hit jobmanager. I'm not sure why Flink SQL Client is > hitting "flink-jobmanager/10.98.253.58:8081". I'd expect either > "flink-jobmanager:8081" or "10.98.253.58:8081" (which should work with my > kubernetes setup). > > I'm using riskfocus's Flink helm chart > <https://github.com/riskfocus/helm-charts-public/tree/master/flink>. > > The last SELECT errors out. I have an environment file that indicates > this "execution.type: batch". My setup works when using DataSet and > DataStream. The jobmanager and taskmanager logs look fine. This seems > like a weird configuration with SQL Client that is either broken with that > Flink helm chart or with SQL Client. > > > Flink SQL> DROP TABLE `default_catalog.mydb.user`; > > [INFO] Table has been removed. > > > Flink SQL> CREATE TABLE `default_catalog.mydb.user` (`platformId` BIGINT, > `userId` STRING) WITH ('connector' = 'filesystem', 'path' = > 's3://mys3bucket/users.csv','format' = 'csv'); > > [INFO] Table has been created. > > > Flink SQL> SELECT * FROM `default_catalog.mydb.user` LIMIT 10; > > *[ERROR] Could not execute SQL statement. Reason:* > > *org.apache.flink.shaded.netty4.io.netty.channel.ConnectTimeoutException: > connection timed out: flink-jobmanager/10.98.253.58:8081 > <http://10.98.253.58:8081>* > > My config logs from jobmanager. > > 2020-09-11 02:33:07,962 INFO > org.apache.flink.contrib.streaming.state.RocksDBStateBackend > [] - Using predefined options: DEFAULT. > > 2020-09-11 02:33:07,962 INFO > org.apache.flink.contrib.streaming.state.RocksDBStateBackend > [] - Using default options factory: > DefaultConfigurableOptionsFactory{configuredOptions={}}. > > 2020-09-11 02:33:08,100 INFO > org.apache.flink.table.client.gateway.local.ProgramDeployer [] - > Submitting job org.apache.flink.streaming.api.graph.StreamGraph@3bd08435 > for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10` > > 2020-09-11 02:33:08,112 INFO org.apache.flink.configuration.Configuration > [] - Config uses fallback configuration key > 'jobmanager.rpc.address' instead of key 'rest.address' > > tail: log/flink--sql-client-flink-jobmanager-0.log: file truncated > > 2020-09-11 02:34:35,848 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > taskmanager.numberOfTaskSlots, 2 > > 2020-09-11 02:34:35,884 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: blob.server.port, 6124 > > 2020-09-11 02:34:35,884 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: taskmanager.rpc.port, 6122 > > 2020-09-11 02:34:35,885 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: jobmanager.heap.size, 1g > > 2020-09-11 02:34:35,886 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > taskmanager.memory.process.size, 1g > > 2020-09-11 02:34:35,886 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.backend, rocksdb > > 2020-09-11 02:34:35,886 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.checkpoints.dir, > file:///flink_state/checkpoints > > 2020-09-11 02:34:35,887 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.savepoints.dir, > file:///flink_state/savepoints > > 2020-09-11 02:34:35,887 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.backend.async, true > > 2020-09-11 02:34:35,887 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.fs.memory-threshold, 1024 > > 2020-09-11 02:34:35,888 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.fs.write-buffer-size, 4096 > > 2020-09-11 02:34:35,888 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.backend.incremental, > true > > 2020-09-11 02:34:35,888 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.local-recovery, true > > 2020-09-11 02:34:35,889 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.checkpoints.num-retained, 1 > > 2020-09-11 02:34:35,889 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > taskmanager.state.local.root-dirs, file:///flink_state/local-recovery > > 2020-09-11 02:34:35,889 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.checkpoint.transfer.thread.num, 1 > > 2020-09-11 02:34:35,890 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.localdir, /flink_state/rocksdb > > 2020-09-11 02:34:35,890 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.options-factory, > org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory > > 2020-09-11 02:34:35,890 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.predefined-options, DEFAULT > > 2020-09-11 02:34:35,891 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.timer-service.factory, HEAP > > 2020-09-11 02:34:35,891 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.ttl.compaction.filter.enabled, false > > 2020-09-11 02:34:35,892 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: jobmanager.rpc.address, > flink-jobmanager > > 2020-09-11 02:34:35,892 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: jobmanager.rpc.port, 6123 > > 2020-09-11 02:34:35,892 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > taskmanager.memory.jvm-metaspace.size, 256mb > > 2020-09-11 02:34:35,893 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: s3a.endpoint, > http://minio:9000 > > 2020-09-11 02:34:35,893 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: s3a.path.style.access, true > > 2020-09-11 02:34:35,894 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: s3a.access-key, > YOURACCESSKEY > > 2020-09-11 02:34:35,894 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: s3a.secret-key, ****** > > 2020-09-11 02:34:35,895 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > s3a.aws.credentials.provider, > org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider > > 2020-09-11 02:34:36,492 INFO org.apache.flink.client.cli.CliFrontend > [] - Loading FallbackYarnSessionCli > > 2020-09-11 02:34:36,498 INFO > org.apache.flink.table.client.gateway.local.LocalExecutor [] - Using > default environment file: file:/opt/flink/conf/sql-client-defaults.yaml > > 2020-09-11 02:34:37,786 INFO org.apache.flink.table.client.SqlClient > [] - Using session environment file: > file:/opt/flink/sql-client-defaults.yaml > > 2020-09-11 02:34:37,931 INFO > org.apache.flink.table.client.config.entries.ExecutionEntry [] - > Property 'execution.restart-strategy.type' not specified. Using default > value: fallback > > 2020-09-11 02:34:43,269 INFO > org.apache.flink.table.client.gateway.local.ExecutionContext > [] - Executor config: {blob.server.port=6124, s3a.secret-key=YOURSECRETKEY, > state.checkpoints.num-retained=1, s3a.access-key=YOURACCESSKEY, > state.backend.rocksdb.options-factory=org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory, > jobmanager.rpc.address=flink-jobmanager, > taskmanager.memory.jvm-metaspace.size=256mb, > state.backend.rocksdb.predefined-options=DEFAULT, > state.savepoints.dir=file:///flink_state/savepoints, > state.backend.fs.memory-threshold=1024, > state.backend.rocksdb.timer-service.factory=HEAP, > execution.savepoint.ignore-unclaimed-state=false, > taskmanager.numberOfTaskSlots=2, state.backend.async=true, > pipeline.classpaths=[], > state.backend.rocksdb.ttl.compaction.filter.enabled=false, > jobmanager.heap.size=1g, taskmanager.memory.process.size=1g, s3a.endpoint= > http://minio:9000, > state.backend.rocksdb.checkpoint.transfer.thread.num=1, > state.backend.local-recovery=true, > state.backend.rocksdb.localdir=/flink_state/rocksdb, > state.backend.incremental=true, execution.target=remote, > taskmanager.rpc.port=6122, jobmanager.rpc.port=6123, > s3a.aws.credentials.provider=org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider, > state.backend.fs.write-buffer-size=4096, execution.attached=true, > execution.shutdown-on-attached-exit=false, > pipeline.jars=[file:/opt/flink/opt/flink-sql-client_2.12-1.11.1.jar], > state.backend=rocksdb, s3a.path.style.access=true, > taskmanager.state.local.root-dirs=file:///flink_state/local-recovery, > state.checkpoints.dir=file:///flink_state/checkpoints} > > 2020-09-11 02:34:43,286 INFO > org.apache.flink.client.deployment.DefaultClusterClientServiceLoader > [] - Could not load factory due to missing dependencies. > > 2020-09-11 02:34:44,079 INFO org.apache.flink.table.client.cli.CliClient > [] - Command history file path: /root/.flink-sql-history > > 2020-09-11 02:35:15,396 INFO > org.apache.flink.contrib.streaming.state.RocksDBStateBackend > [] - Using predefined options: DEFAULT. > > 2020-09-11 02:35:15,397 INFO > org.apache.flink.contrib.streaming.state.RocksDBStateBackend > [] - Using default options factory: > DefaultConfigurableOptionsFactory{configuredOptions={}}. > > 2020-09-11 02:35:17,529 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > taskmanager.numberOfTaskSlots, 2 > > 2020-09-11 02:35:17,530 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: blob.server.port, 6124 > > 2020-09-11 02:35:17,530 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: taskmanager.rpc.port, 6122 > > 2020-09-11 02:35:17,530 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: jobmanager.heap.size, 1g > > 2020-09-11 02:35:17,530 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > taskmanager.memory.process.size, 1g > > 2020-09-11 02:35:17,530 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.backend, rocksdb > > 2020-09-11 02:35:17,530 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.checkpoints.dir, > file:///flink_state/checkpoints > > 2020-09-11 02:35:17,531 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.savepoints.dir, > file:///flink_state/savepoints > > 2020-09-11 02:35:17,531 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.backend.async, true > > 2020-09-11 02:35:17,531 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.fs.memory-threshold, 1024 > > 2020-09-11 02:35:17,531 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.fs.write-buffer-size, 4096 > > 2020-09-11 02:35:17,531 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: state.backend.incremental, > true > > 2020-09-11 02:35:17,531 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.local-recovery, true > > 2020-09-11 02:35:17,532 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.checkpoints.num-retained, 1 > > 2020-09-11 02:35:17,532 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > taskmanager.state.local.root-dirs, file:///flink_state/local-recovery > > 2020-09-11 02:35:17,532 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.checkpoint.transfer.thread.num, 1 > > 2020-09-11 02:35:17,532 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.localdir, /flink_state/rocksdb > > 2020-09-11 02:35:17,532 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.options-factory, > org.apache.flink.contrib.streaming.state.DefaultConfigurableOptionsFactory > > 2020-09-11 02:35:17,532 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.predefined-options, DEFAULT > > 2020-09-11 02:35:17,532 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.timer-service.factory, HEAP > > 2020-09-11 02:35:17,533 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > state.backend.rocksdb.ttl.compaction.filter.enabled, false > > 2020-09-11 02:35:17,533 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: jobmanager.rpc.address, > flink-jobmanager > > 2020-09-11 02:35:17,533 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: jobmanager.rpc.port, 6123 > > 2020-09-11 02:35:17,533 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > taskmanager.memory.jvm-metaspace.size, 256mb > > 2020-09-11 02:35:17,533 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: s3a.endpoint, > http://minio:9000 > > 2020-09-11 02:35:17,534 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: s3a.path.style.access, true > > 2020-09-11 02:35:17,534 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: s3a.access-key, > YOURACCESSKEY > > 2020-09-11 02:35:17,534 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: s3a.secret-key, ****** > > 2020-09-11 02:35:17,535 INFO > org.apache.flink.configuration.GlobalConfiguration > [] - Loading configuration property: > s3a.aws.credentials.provider, > org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider > > 2020-09-11 02:35:18,047 INFO > org.apache.flink.table.client.gateway.local.ProgramDeployer [] - > Submitting job org.apache.flink.streaming.api.graph.StreamGraph@2f95653f > for query default: SELECT * FROM `default_catalog.mydb.user` LIMIT 10` > > 2020-09-11 02:35:18,382 INFO org.apache.flink.configuration.Configuration > [] - Config uses fallback configuration key > 'jobmanager.rpc.address' instead of key 'rest.address' > > 2020-09-11 02:41:39,539 WARN org.apache.flink.runtime.rest.RestClient > [] - Rest endpoint shutdown failed. > > java.util.concurrent.TimeoutException: null >
