Hello, I'm getting an exception when running a NiFi flow, using the PutHiveQL processor:
10:12:58 UTC ERROR 5f9d00fe-e6dd-3c05-8698-50cab74ee142 nifi-node-0:8080 PutHiveQL[id=5f9d00fe-e6dd-3c05-8698-50cab74ee142] Failed to process session due to Failed to process StandardFlowFileRecord[uuid=8bd48767-dbb3-42c1-8492-4063e1aee6ec,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1579790709540-2, container=default, section=2], offset=448724, length=182],offset=0,name=3613831121596506,size=182] due to java.sql.SQLException: org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.adl.AdlFileSystem not found);: org.apache.nifi.processor.exception.ProcessException: Failed to process StandardFlowFileRecord[uuid=8bd48767-dbb3-42c1-8492-4063e1aee6ec,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1579790709540-2, container=default, section=2], offset=448724, length=182],offset=0,name=3613831121596506,size=182] due to java.sql.SQLException: org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.adl.AdlFileSystem not found); I've posted this question on the NiFi users list, but maybe you can help me a bit (I'm new both to Hive and NiFi). I had a NiFi cluster with a flow using PutHDFS, and after that, I used PutHiveQL to addPartition. Now I've setup another NiFi cluster, with git as the flow provider, and we replaced HDFS with Minio, so instead using PutParquet/PutHDFS, I'm using PutS3Object. AFAIK, PutHiveQL is talking to our Thrift server, so no need for Hadoop or AzureDataLake dependencies. However, I'm getting this. Both NiFi clusters are deployed on a custom Kubernetes installation on Azure. I'm kind of lost here, so any help is much appreciated. Thanks Juan A. Fabián Simón Data Engineer Alstom Calle Martínez Villergas 49, ed. V - 28027 Madrid - Spain Office: +34 91 384 89 00 Email: <mailto:juan-antonio.fab...@alstomgroup.com> juan-antonio.fab...@alstomgroup.com www.alstom.com<http://www.alstom.com> [cid:850c3903-2713-45c3-9040-c2f64d22bc89] [cid:3539e2fe-0e83-445a-be92-2063029e4590] <https://twitter.com/Alstom> [cid:fa891413-1d00-4c29-8789-4fe7149b2bef] <https://www.linkedin.com/company/alstom/> [cid:34c4d7b6-85be-4793-8910-9805a338456b] <https://www.facebook.com/ALSTOM/> [cid:c542d5f3-d10b-4fda-8c03-fdc0d155e987] <https://www.instagram.com/alstom> [cid:f6b0d063-0c70-4343-85ef-57dc6b720b3d] <https://www.youtube.com/user/Alstom> ________________________________ CONFIDENTIALITY : This e-mail and any attachments are confidential and may be privileged. If you are not a named recipient, please notify the sender immediately and do not disclose the contents to another person, use it for any purpose or store or copy the information in any medium.
