On Tue, Jun 07, 2022 at 03:26:43PM +0800, czc...@gmail.com wrote:
> On Mon, Jun 06, 2022 at 10:42:08AM +0800, Shengkai Fang wrote:
> > Hi. In my experience, the step to debug classloading problems are as
> > follows:
>
> Thanks for the help. We get the following log when using
> `-verbose:class`:
On Mon, Jun 06, 2022 at 10:42:08AM +0800, Shengkai Fang wrote:
> Hi. In my experience, the step to debug classloading problems are as
> follows:
Thanks for the help. We get the following log when using
`-verbose:class`:
[2.074s][info][class,load] org.apache.hadoop.fs.FileSystem source:
file:/o
On Thu, Jun 02, 2022 at 06:23:20PM +0800, Qingsheng Ren wrote:
> Thanks for the input ChangZhuo.
>
> Could you check if the configuration "classloader.resolve-order” is
> set to “parent-first” in your Flink 1.14 cluster? I didn’t notice any
> changes related to the user code classloader in Flink 1
I believe the change between Flink 1.14 and Flink 1.15 has been the
addition of a RecoverableWriter for GCS [1]
Perhaps this is the reason for this failure?
Best regards, Martijn
[1] https://issues.apache.org/jira/browse/FLINK-11838
Op do 2 jun. 2022 om 12:24 schreef Qingsheng Ren :
> Thanks f
Thanks for the input ChangZhuo.
Could you check if the configuration "classloader.resolve-order” is
set to “parent-first” in your Flink 1.14 cluster? I didn’t notice any
changes related to the user code classloader in Flink 1.15. If my
assumption is correct, you package the gcs-connector into your
On Thu, Jun 02, 2022 at 11:17:19AM +0800, Qingsheng Ren wrote:
> Hi ChangZhuo,
>
> I assume it’s a classloading issue but I can’t track down to the root cause
> in code. Would you mind sharing the entire exception stack and some JM/TM
> logs related to file system?
The following is exception lo
Hi ChangZhuo,
I assume it’s a classloading issue but I can’t track down to the root cause in
code. Would you mind sharing the entire exception stack and some JM/TM logs
related to file system?
Best regards,
Qingsheng
> On Jun 2, 2022, at 09:08, ChangZhuo Chen (陳昌倬) wrote:
>
> Hi,
>
> We u
Hi,
We use GCS as storage, and have the following functions to list files in
GCS path for Flink batch mode to buidl states:
def listPath(p: String): Seq[String] = {
val path = new Path(p)
val fs = path.getFileSystem(new Configuration())
fs.listStatus(path) match {
case null =
