Just to close this thread I found the cause of the problem: looking into
the code of the mysql connector the value of
PropertyDefinitions.SYSP_disableAbandonedConnectionCleanup
is "com.mysql.cj.disableAbandonedConnectionCleanup" and not
"com.mysql.disableAbandonedConnectionCleanup" as stated in [1]
No magic for JVM properties afaik (and I just looked in the code base for
the most obvious candidates). There is also nothing to gain from
overwriting properties.
I'm also certain that it should work as it's used in most secured setups to
inject keys/keytabs.
What happens if you execute the Flink
Yes, that's what is surprising..I already did a remote debug on the TM and
that property is not read..but that's really weird..could it be that the
JVM properties gets cleared before invoking the tasks?
Il ven 20 nov 2020, 12:50 Arvid Heise ha scritto:
> All looks good and as it should be.
>
> C
All looks good and as it should be.
Can you do a remote debugging session to the tm once more and check
Boolean.getBoolean("com.mysql.disableAbandonedConnectionCleanup")
There is no magic involved in System properties in Flink. If the property
is set on the process, the configuration works. If it
I've just tested the following code in a java class and the property
(-Dcom.mysql.cj.disableAbandonedConnectionCleanup=true) is read correctly
and the abandonedConnectionCleanupDisabled does not initialize
the cleanupThreadExecutorService (that in my other test was causing a
dynamic classloading m
no no I didn't relocate any class related to jdbc
Il ven 20 nov 2020, 10:02 Arvid Heise ha scritto:
> I was particularly asking if you relocate classes. Since the property name
> looks like a class name, it could have been changed as well. Could you
> check the value of
> PropertyDefinitions.SYS
I was particularly asking if you relocate classes. Since the property name
looks like a class name, it could have been changed as well. Could you
check the value of
PropertyDefinitions.SYSP_disableAbandonedConnectionCleanup in your final
jar?
On Fri, Nov 20, 2020 at 9:35 AM Flavio Pompermaier
wro
the mysql connector is put in the client classpath and in the Flink lib
dir. When i debugged remotely the AbandonedConnectionCleanupThread was
initialized at the first run of the job by the taskmamager. Today I'll try
to run the mysql connector in a standalone java app to see if the property
is rea
Hi Flavio,
if it arrives in the java process then you are doing everything right
already (or almost).
Are you shading the mysql connector? I'm suspecting that the property also
get shaded then. You could decompile your jar to be sure. Have you verified
that this is working as intended without Fli
the properties arrives to the task manager because I can see them in the
java process (using ps aux)..or donyoubmean some special line of code?
Il gio 19 nov 2020, 20:53 Arvid Heise ha scritto:
> Hi Flavio,
>
> you are right, all looks good.
>
> Can you please verify if the properties arrived at
Hi Flavio,
you are right, all looks good.
Can you please verify if the properties arrived at the task manager in the
remote debugger session? For example, you could check the JVisualVM
Overview tab.
On Thu, Nov 19, 2020 at 8:38 PM Flavio Pompermaier
wrote:
> At the moment I use a standalone cl
At the moment I use a standalone cluster, isn't using env.java.opts the
right way to do it?
Il gio 19 nov 2020, 20:11 Arvid Heise ha scritto:
> Hi Flavio,
>
> -D afaik passes only the system property to the entry point (client or
> jobmanager depending on setup), while you probably want to have
Hi Flavio,
-D afaik passes only the system property to the entry point (client or
jobmanager depending on setup), while you probably want to have it on the
task managers.
The specific options to pass it to the task managers depend on the way you
deploy. -yD for yarn for example. For docker or k8s
Hi to all,
while trying to solve a leak with dynamic class loading I found out
that mysql connector creates an AbandonedConnectionCleanupThread that
is retained in the ChildFirstClassLoader..from version 8.0.22 there's
the possibility to inhibit this thread passing the system property
com.mysql.dis
14 matches
Mail list logo
