Hi DEROCCO, I think you need to check whether there is a hadoop-azure jar file in the classpath of your flink job. From an error message '*Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider not found.*', your flink job may be missing this package.
Best, Shammon FY On Fri, May 5, 2023 at 4:40 AM DEROCCO, CHRISTOPHER <cd9...@att.com> wrote: > > > I receive the error: *Caused by: java.lang.ClassNotFoundException: Class > org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider not found.* > > I’m using flink 1.16 running in Azure Kubernetes using the Flink Apache > Kubernetes Operator. > > I have the following specified in the spec.flinkConfiguration: as per the > Apache Kubernetes operator documentation. > > > > fs.azure.createRemoteFileSystemDuringInitialization: "true" > > fs.azure.account.auth.type.storageaccountname.dfs.core.windows.net: > OAuth > > fs.azure.account.oauth.provider.type.<storageaccountname>. > dfs.core.windows.net: > org.apache.hadoop.fs.azurebfs.oauth2.MsiTokenProvider > > fs.azure.account.oauth2.msi.tenant. <storageaccountname>. > dfs.core.windows.net: <MY TENANT ID> > > fs.azure.account.oauth2.client.id. <storageaccountname>. > dfs.core.windows.net: <MY CLIENT ID of VM> > > fs.azure.account.oauth2.client.endpoint. <storageaccountname>. > dfs.core.windows.net: https://login.microsoftonline.com/<MY TENANT > ID>/oauth2/token > > > > I also have this specified in the container environment variables. > > - name: ENABLE_BUILT_IN_PLUGINS > > value: flink-azure-fs-hadoop-1.16.1.jar > > > > I think I’m missing a configuration step because the MsiTokenProvider > class is not found based on the logs. Any help would be appreciated. > > > > > > *Chris deRocco* > > Senior – Cybersecurity > > Chief Security Office | STORM Threat Analytics > > > > *AT&T* > > Middletown, NJ > > Phone: 732-639-9342 > > Email: cd9...@att.com > > >
