Re: [Third-party Tool] Flink memory calculator

2020-07-07 Thread Yangze Guo
Hi, there, As Flink 1.11.0 released, we provide a new calculator[1] for this version. Feel free to try it and any feedback or suggestion is welcomed! [1] https://github.com/KarmaGYZ/flink-memory-calculator/blob/master/calculator-1.11.sh Best, Yangze Guo On Wed, Apr 1, 2020 at 9:45 PM Yangze Gu

Re: [Third-party Tool] Flink memory calculator

2020-04-01 Thread Yangze Guo
@Marta Thanks for the tip! I'll do that. Best, Yangze Guo On Wed, Apr 1, 2020 at 8:05 PM Marta Paes Moreira wrote: > > Hey, Yangze. > > I'd like to suggest that you submit this tool to Flink Community Pages [1]. > That way it can get more exposure and it'll be easier for users to find it. > > T

Re: [Third-party Tool] Flink memory calculator

2020-04-01 Thread Marta Paes Moreira
Hey, Yangze. I'd like to suggest that you submit this tool to Flink Community Pages [1]. That way it can get more exposure and it'll be easier for users to find it. Thanks for your contribution! [1] https://flink-packages.org/ On Tue, Mar 31, 2020 at 9:09 AM Yangze Guo wrote: > Hi, there. > >

Re: [Third-party Tool] Flink memory calculator

2020-03-31 Thread Yangze Guo
Hi, there. In the latest version, the calculator supports dynamic options. You could append all your dynamic options to the end of "bin/calculator.sh [-h]". Since "-tm" will be deprecated eventually, please replace it with "-Dtaskmanager.memory.process.size=". Best, Yangze Guo On Mon, Mar 30, 20

Re: [Third-party Tool] Flink memory calculator

2020-03-29 Thread Xintong Song
Hi Jeff, I think the purpose of this tool it to allow users play with the memory configurations without needing to actually deploy the Flink cluster or even have a job. For sanity checks, we currently have them in the start-up scripts (for standalone clusters) and resource managers (on K8s/Yarn/Me

Re: [Third-party Tool] Flink memory calculator

2020-03-29 Thread Yangze Guo
Thanks for your feedbacks, @Xintong and @Jeff. @Jeff I think it would always be good to leverage exist logic in Flink, such as JobListener. However, this calculator does not only target to check the conflict, it also targets to provide the calculating result to user before the job is actually depl

Re: [Third-party Tool] Flink memory calculator

2020-03-29 Thread Jeff Zhang
Hi Yangze, Does this tool just parse the configuration in flink-conf.yaml ? Maybe it could be done in JobListener [1] (we should enhance it via adding hook before job submission), so that it could all the cases (e.g. parameters coming from command line) [1] https://github.com/apache/flink/blob/m

Re: [Third-party Tool] Flink memory calculator

2020-03-29 Thread Xintong Song
Thanks Yangze, I've tried the tool and I think its very helpful. Thank you~ Xintong Song On Mon, Mar 30, 2020 at 9:40 AM Yangze Guo wrote: > Hi, Yun, > > I'm sorry that it currently could not handle it. But I think it is a > really good idea and that feature would be added to the next versi

Re: [Third-party Tool] Flink memory calculator

2020-03-29 Thread Yangze Guo
Hi, Yun, I'm sorry that it currently could not handle it. But I think it is a really good idea and that feature would be added to the next version. Best, Yangze Guo On Mon, Mar 30, 2020 at 12:21 AM Yun Tang wrote: > > Very interesting and convenient tool, just a quick question: could this tool

Re: [Third-party Tool] Flink memory calculator

2020-03-29 Thread Yun Tang
Very interesting and convenient tool, just a quick question: could this tool also handle deployment cluster commands like "-tm" mixed with configuration in `flink-conf.yaml` ? Best Yun Tang From: Yangze Guo Sent: Friday, March 27, 2020 18:00 To: user ; user...@f