Hi all, I’m running into the exact same issue, though with Flink 1.15.1. I get a ClassCastException with HoodieWrapperFileSystem being loaded from two different ChildFirstClassLoader instances:
Caused by: java.lang.ClassCastException: class shaded.org.apache.hudi.common.fs.HoodieWrapperFileSystem cannot be cast to class shaded.org.apache.hudi.common.fs.HoodieWrapperFileSystem (shaded.org.apache.hudi.common.fs.HoodieWrapperFileSystem is in unnamed module of loader org.apache.flink.util.ChildFirstClassLoader @52deeb14; shaded.org.apache.hudi.common.fs.HoodieWrapperFileSystem is in unnamed module of loader org.apache.flink.util.ChildFirstClassLoader @5dab96b9) Switching to parent-first doesn’t solve the problem. Did this ever get resolved? When running w/the MiniCluster this doesn’t happen. Regards, — Ken > On Jan 31, 2022, at 5:34 AM, Till Rohrmann <trohrm...@apache.org> wrote: > > Hi Zhiping, > > I am pulling in Timo who knows the sql client probably best. He might be > able to help you. In general, this sounds like a bug to me. Maybe you can > provide us with the full stack trace and maybe even the debug logs of the > failing run. > > Cheers, > Till > > On Sun, Jan 30, 2022 at 4:21 AM Zhiping Wu <stayras...@gmail.com> wrote: > >> Hi there, >> >> I'm trying to consult whether a flink class loader(ParentFirstClassLoader >> or ChildFirstClassLoader) must be a singleton? >> >> I met a problem with the below error message by using sql-client to submit >> sql jobs to a standalone session model cluster with Flink 1.13.3. >> I also tried ParentFirstClassLoader & ChildFirstClassLoader, both of them >> will meet this problem. >> >> Caused by: java.lang.ClassCastException: >> class org.apache.hudi.common.fs.HoodieWrapperFileSystem cannot be cast to >> class org.apache.hudi.common.fs.HoodieWrapperFileSystem >> (org.apache.hudi.common.fs.HoodieWrapperFileSystem >> is in unnamed module of loader >> org.apache.flink.runtime.execution.librarycache. >> FlinkUserCodeClassLoaders$ParentFirstClassLoader @*384d2142*; >> org.apache.hudi.common.fs.HoodieWrapperFileSystem >> is in unnamed module of loader >> org.apache.flink.runtime.execution.librarycache. >> FlinkUserCodeClassLoaders$ParentFirstClassLoader @*35e21dd1*) >> >> From this documentation, it shows I might have two versions of the class >> which are different in application code and flink libraries. >> But in my case, Flink should don't contains HoodieWrapperFileSystem class, >> and I didn't add the hudi-flink-bundle jar to flink lib folder, instead, i >> used sql-client.sh embedded -j xxxx/huid-flink-bundle-xxx.jar. >> >> So I'm wondering if the root reason is that they are loaded by two >> classloader instances? And it seems that FlinkUserCodeClassLoaders is a >> singleton, but ParentFirstClassLoader and ChildFirstClassLoader aren't from >> flink source code. >> -------------------------- Ken Krugler http://www.scaleunlimited.com Custom big data solutions Flink, Pinot, Solr, Elasticsearch
