Hi, you are missing the Hadoop libraries, hence there is no hdfs support.
In Flink 1.10 and earlier, you would simply copy flink-shaded-hadoop-2-uber[1] into your opt/ folder. However, since Flink 1.11, we recommend to install Hadoop and point to it with HADOOP_CLASSPATH. Now, the latter approach does not seem to be terribly meaningful to K8s. You can still use the old flink-shaded-hadoop-2-uber for now but it's better to have a different way in the future. @Andrey Zagrebin <[email protected]> , do you know what's the current recommended approach? Should we release a light-weight hdfs filesystem to make it symmetric to the other filesystems? [1] https://ci.apache.org/projects/flink/flink-docs-release-1.11/ops/deployment/hadoop.html#using-flink-shaded-hadoop-2-uber-jar-for-resolving-dependency-conflicts-legacy On Wed, Sep 30, 2020 at 9:02 AM superainbower <[email protected]> wrote: > And I got this error log > > Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: > Hadoop is not in the classpath/dependencies. > Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: > Could not find a file system implementation for scheme 'hdfs'. The scheme > is not directly supported by Flink and no Hadoop file system to support > this scheme could be loaded. For a full list of supported file systems, > please see > https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/. > > > superainbower > [email protected] > > <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=superainbower&uid=superainbower%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22superainbower%40163.com%22%5D> > 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 > > On 09/30/2020 14:48，superainbower<[email protected]> > <[email protected]> wrote： > > Hi > How to configure statebackend when I deploy flink on k8s , I just add > the following to flink-conf.yaml, but it doesn’t work > > state.backend: rocksdb > state.checkpoints.dir: hdfs://slave2:8020/flink/checkpoints > state.savepoints.dir: hdfs://slave2:8020/flink/savepoints > state.backend.incremental: true > > superainbower > [email protected] > > <https://maas.mail.163.com/dashi-web-extend/html/proSignature.html?ftlId=1&name=superainbower&uid=superainbower%40163.com&iconUrl=https%3A%2F%2Fmail-online.nosdn.127.net%2Fqiyelogo%2FdefaultAvatar.png&items=%5B%22superainbower%40163.com%22%5D> > 签名由 网易邮箱大师 <https://mail.163.com/dashi/dlpro.html?from=mail81> 定制 > > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
