the mysql connector is put in the client classpath and in the Flink lib dir. When i debugged remotely the AbandonedConnectionCleanupThread was initialized at the first run of the job by the taskmamager. Today I'll try to run the mysql connector in a standalone java app to see if the property is read correctly or not.
Il ven 20 nov 2020, 07:52 Arvid Heise <ar...@ververica.com> ha scritto: > Hi Flavio, > > if it arrives in the java process then you are doing everything right > already (or almost). > > Are you shading the mysql connector? I'm suspecting that the property also > get shaded then. You could decompile your jar to be sure. Have you verified > that this is working as intended without Flink? > > On Thu, Nov 19, 2020 at 9:19 PM Flavio Pompermaier <pomperma...@okkam.it> > wrote: > >> the properties arrives to the task manager because I can see them in the >> java process (using ps aux)..or donyoubmean some special line of code? >> >> Il gio 19 nov 2020, 20:53 Arvid Heise <ar...@ververica.com> ha scritto: >> >>> Hi Flavio, >>> >>> you are right, all looks good. >>> >>> Can you please verify if the properties arrived at the task manager in >>> the remote debugger session? For example, you could check the JVisualVM >>> Overview tab. >>> >>> On Thu, Nov 19, 2020 at 8:38 PM Flavio Pompermaier <pomperma...@okkam.it> >>> wrote: >>> >>>> At the moment I use a standalone cluster, isn't using env.java.opts the >>>> right way to do it? >>>> >>>> Il gio 19 nov 2020, 20:11 Arvid Heise <ar...@ververica.com> ha scritto: >>>> >>>>> Hi Flavio, >>>>> >>>>> -D afaik passes only the system property to the entry point (client or >>>>> jobmanager depending on setup), while you probably want to have it on the >>>>> task managers. >>>>> >>>>> The specific options to pass it to the task managers depend on the way >>>>> you deploy. -yD for yarn for example. For docker or k8s, you would use >>>>> env. >>>>> >>>>> On Wed, Nov 18, 2020 at 10:20 PM Flavio Pompermaier < >>>>> pomperma...@okkam.it> wrote: >>>>> >>>>>> Hi to all, >>>>>> while trying to solve a leak with dynamic class loading I found out >>>>>> that mysql connector creates an AbandonedConnectionCleanupThread that >>>>>> is retained in the ChildFirstClassLoader..from version 8.0.22 there's >>>>>> the possibility to inhibit this thread passing the system property >>>>>> com.mysql.disableAbandonedConnectionCleanup=true [1] (I put the mysql >>>>>> jar in the lib folder). >>>>>> >>>>>> I tried to set in the flink-conf.yml >>>>>> env.java.opts: "-Dcom.mysql.disableAbandonedConnectionCleanup=true" >>>>>> >>>>>> but the property does not produce the desired effect in the static >>>>>> section of such a thread [2] (I verified that attaching the remote >>>>>> debugger to the task manager). >>>>>> >>>>>> How can I fix this problem? >>>>>> >>>>>> [1] >>>>>> https://dev.mysql.com/doc/relnotes/connector-j/8.0/en/news-8-0-22.html >>>>>> [2] >>>>>> public class AbandonedConnectionCleanupThread implements Runnable {8898 >>>>>> 7324 0768 >>>>>> private static boolean abandonedConnectionCleanupDisabled = >>>>>> >>>>>> Boolean.getBoolean(PropertyDefinitions.SYSP_disableAbandonedConnectionCleanup); >>>>>> >>>>>> static { >>>>>> if (abandonedConnectionCleanupDisabled) { >>>>>> cleanupThreadExecutorService = null; >>>>>> } else { >>>>>> cleanupThreadExecutorService = >>>>>> Executors.newSingleThreadExecutor(r -> {....} >>>>>> } >>>>>> } >>>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Arvid Heise | Senior Java Developer >>>>> >>>>> <https://www.ververica.com/> >>>>> >>>>> Follow us @VervericaData >>>>> >>>>> -- >>>>> >>>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >>>>> Conference >>>>> >>>>> Stream Processing | Event Driven | Real Time >>>>> >>>>> -- >>>>> >>>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >>>>> >>>>> -- >>>>> Ververica GmbH >>>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >>>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, >>>>> Ji (Toni) Cheng >>>>> >>>> >>> >>> -- >>> >>> Arvid Heise | Senior Java Developer >>> >>> <https://www.ververica.com/> >>> >>> Follow us @VervericaData >>> >>> -- >>> >>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >>> Conference >>> >>> Stream Processing | Event Driven | Real Time >>> >>> -- >>> >>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >>> >>> -- >>> Ververica GmbH >>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji >>> (Toni) Cheng >>> >> > > -- > > Arvid Heise | Senior Java Developer > > <https://www.ververica.com/> > > Follow us @VervericaData > > -- > > Join Flink Forward <https://flink-forward.org/> - The Apache Flink > Conference > > Stream Processing | Event Driven | Real Time > > -- > > Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany > > -- > Ververica GmbH > Registered at Amtsgericht Charlottenburg: HRB 158244 B > Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji > (Toni) Cheng >
