I was particularly asking if you relocate classes. Since the property name looks like a class name, it could have been changed as well. Could you check the value of PropertyDefinitions.SYSP_disableAbandonedConnectionCleanup in your final jar?
On Fri, Nov 20, 2020 at 9:35 AM Flavio Pompermaier <pomperma...@okkam.it> wrote: > the mysql connector is put in the client classpath and in the Flink lib > dir. When i debugged remotely the AbandonedConnectionCleanupThread was > initialized at the first run of the job by the taskmamager. Today I'll try > to run the mysql connector in a standalone java app to see if the property > is read correctly or not. > > Il ven 20 nov 2020, 07:52 Arvid Heise <ar...@ververica.com> ha scritto: > >> Hi Flavio, >> >> if it arrives in the java process then you are doing everything right >> already (or almost). >> >> Are you shading the mysql connector? I'm suspecting that the property >> also get shaded then. You could decompile your jar to be sure. Have you >> verified that this is working as intended without Flink? >> >> On Thu, Nov 19, 2020 at 9:19 PM Flavio Pompermaier <pomperma...@okkam.it> >> wrote: >> >>> the properties arrives to the task manager because I can see them in the >>> java process (using ps aux)..or donyoubmean some special line of code? >>> >>> Il gio 19 nov 2020, 20:53 Arvid Heise <ar...@ververica.com> ha scritto: >>> >>>> Hi Flavio, >>>> >>>> you are right, all looks good. >>>> >>>> Can you please verify if the properties arrived at the task manager in >>>> the remote debugger session? For example, you could check the JVisualVM >>>> Overview tab. >>>> >>>> On Thu, Nov 19, 2020 at 8:38 PM Flavio Pompermaier < >>>> pomperma...@okkam.it> wrote: >>>> >>>>> At the moment I use a standalone cluster, isn't using env.java.opts >>>>> the right way to do it? >>>>> >>>>> Il gio 19 nov 2020, 20:11 Arvid Heise <ar...@ververica.com> ha >>>>> scritto: >>>>> >>>>>> Hi Flavio, >>>>>> >>>>>> -D afaik passes only the system property to the entry point (client >>>>>> or jobmanager depending on setup), while you probably want to have it on >>>>>> the task managers. >>>>>> >>>>>> The specific options to pass it to the task managers depend on the >>>>>> way you deploy. -yD for yarn for example. For docker or k8s, you would >>>>>> use >>>>>> env. >>>>>> >>>>>> On Wed, Nov 18, 2020 at 10:20 PM Flavio Pompermaier < >>>>>> pomperma...@okkam.it> wrote: >>>>>> >>>>>>> Hi to all, >>>>>>> while trying to solve a leak with dynamic class loading I found out >>>>>>> that mysql connector creates an AbandonedConnectionCleanupThread that >>>>>>> is retained in the ChildFirstClassLoader..from version 8.0.22 there's >>>>>>> the possibility to inhibit this thread passing the system property >>>>>>> com.mysql.disableAbandonedConnectionCleanup=true [1] (I put the mysql >>>>>>> jar in the lib folder). >>>>>>> >>>>>>> I tried to set in the flink-conf.yml >>>>>>> env.java.opts: "-Dcom.mysql.disableAbandonedConnectionCleanup=true" >>>>>>> >>>>>>> but the property does not produce the desired effect in the static >>>>>>> section of such a thread [2] (I verified that attaching the remote >>>>>>> debugger to the task manager). >>>>>>> >>>>>>> How can I fix this problem? >>>>>>> >>>>>>> [1] >>>>>>> https://dev.mysql.com/doc/relnotes/connector-j/8.0/en/news-8-0-22.html >>>>>>> [2] >>>>>>> public class AbandonedConnectionCleanupThread implements Runnable {8898 >>>>>>> 7324 0768 >>>>>>> private static boolean abandonedConnectionCleanupDisabled = >>>>>>> >>>>>>> Boolean.getBoolean(PropertyDefinitions.SYSP_disableAbandonedConnectionCleanup); >>>>>>> >>>>>>> static { >>>>>>> if (abandonedConnectionCleanupDisabled) { >>>>>>> cleanupThreadExecutorService = null; >>>>>>> } else { >>>>>>> cleanupThreadExecutorService = >>>>>>> Executors.newSingleThreadExecutor(r -> {....} >>>>>>> } >>>>>>> } >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Arvid Heise | Senior Java Developer >>>>>> >>>>>> <https://www.ververica.com/> >>>>>> >>>>>> Follow us @VervericaData >>>>>> >>>>>> -- >>>>>> >>>>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >>>>>> Conference >>>>>> >>>>>> Stream Processing | Event Driven | Real Time >>>>>> >>>>>> -- >>>>>> >>>>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >>>>>> >>>>>> -- >>>>>> Ververica GmbH >>>>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >>>>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, >>>>>> Ji (Toni) Cheng >>>>>> >>>>> >>>> >>>> -- >>>> >>>> Arvid Heise | Senior Java Developer >>>> >>>> <https://www.ververica.com/> >>>> >>>> Follow us @VervericaData >>>> >>>> -- >>>> >>>> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >>>> Conference >>>> >>>> Stream Processing | Event Driven | Real Time >>>> >>>> -- >>>> >>>> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >>>> >>>> -- >>>> Ververica GmbH >>>> Registered at Amtsgericht Charlottenburg: HRB 158244 B >>>> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji >>>> (Toni) Cheng >>>> >>> >> >> -- >> >> Arvid Heise | Senior Java Developer >> >> <https://www.ververica.com/> >> >> Follow us @VervericaData >> >> -- >> >> Join Flink Forward <https://flink-forward.org/> - The Apache Flink >> Conference >> >> Stream Processing | Event Driven | Real Time >> >> -- >> >> Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany >> >> -- >> Ververica GmbH >> Registered at Amtsgericht Charlottenburg: HRB 158244 B >> Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji >> (Toni) Cheng >> > -- Arvid Heise | Senior Java Developer <https://www.ververica.com/> Follow us @VervericaData -- Join Flink Forward <https://flink-forward.org/> - The Apache Flink Conference Stream Processing | Event Driven | Real Time -- Ververica GmbH | Invalidenstrasse 115, 10115 Berlin, Germany -- Ververica GmbH Registered at Amtsgericht Charlottenburg: HRB 158244 B Managing Directors: Timothy Alexander Steinert, Yip Park Tung Jason, Ji (Toni) Cheng
