Re: Flink REST API async?

2017-11-22 Thread Francisco Gonzalez
Sorry guys, in the previous message, when I talked about the task managers performance, I meant *Jobmanager* performance Francisco Gonzalez wrote > Hi guys, > > After investigating a bit more about this topic, we found a solution > adding > a small change in the Flink-1.

Re: Flink REST API async?

2017-11-22 Thread Francisco Gonzalez
Hi guys, After investigating a bit more about this topic, we found a solution adding a small change in the Flink-1.3.2 source code. We found that the issue occurred when different threads tried to build the Tuple2 object at the same time (due to they use the static ExecutionEnvironmnet variable

Re: Flink REST API async?

2017-10-19 Thread Francisco Gonzalez Barea
erface). This should solve the problem, but unfortunately it doesn't solve your current problem. Best, Aljoscha On 8. Aug 2017, at 10:26, Francisco Gonzalez Barea mailto:francisco.gonza...@piksel.com>> wrote: Aha ok… Thanks for your answer Eron. Regards On 7 Aug 2017, at 1

Re: Problem in Flink 1.3.2 with Mesos task managers offers

2017-09-20 Thread Francisco Gonzalez Barea
ay adjust the two minute timeout to something lower as follows. In the `MesosFlinkResourceManager` class, edit the `createOptimizer` method to call `withLeaseOfferExpirySecs` on the `TaskScheduler.Builder` object. Let us know if that helps and we'll make the timeout configurable. -Eron

Problem in Flink 1.3.2 with Mesos task managers offers

2017-09-19 Thread Francisco Gonzalez Barea
Hello guys, We have a flink 1.3.2 session deployed from Marathon json to Mesos with some of the following parameters as environment variables: "flink_mesos.initial-tasks": "8", "flink_mesos.resourcemanager.tasks.mem": "4096", And other environment variables including zookeeper, etc. The mesos

Query Rpc address/port remotely?

2017-08-17 Thread Francisco Gonzalez Barea
Hey guys! I´ve got a new question. Having a Flink v1.3.0 running on Mesos, is there any remotely way (rest, or another) to query which is the rpc address and port to connect that Flink via akka? We´ve been taking a look at the rest API but the config endpoint doesn’t seem to provide this info

Re: Flink REST API async?

2017-08-08 Thread Francisco Gonzalez Barea
cution environment that the program obtains from `ExecutionEnvironment.getExecutionEnvironment()`. From the stack trace it appears that two main methods are executing simultaneously and one is corrupting the other. On Mon, Aug 7, 2017 at 8:21 AM, Francisco Gonzalez Barea mailto:francisco

Flink REST API async?

2017-08-07 Thread Francisco Gonzalez Barea
Hi there! We are doing some POCs submitting jobs remotely to Flink. We tried with Flink CLI and now we´re testing the Rest API. So the point is that when we try to execute a set of requests in an async way (using CompletableFutures) only a couple of them run successfully. For the rest we get t

Re: Flink CLI cannot submit job to Flink on Mesos

2017-08-01 Thread Francisco Gonzalez Barea
lease-1.3/setup/jobmanager_high_availability.html#config-file-flink-confyaml> On 1 Aug 2017, at 10:24, Francisco Gonzalez Barea mailto:francisco.gonza...@piksel.com>> wrote: Hi Stephan, So, do you mean to remove the “-m” param from the flink CLI call? And on the other hand, that I

Re: Flink CLI cannot submit job to Flink on Mesos

2017-08-01 Thread Francisco Gonzalez Barea
tephan On Mon, Jul 31, 2017 at 6:27 PM, Francisco Gonzalez Barea mailto:francisco.gonza...@piksel.com>> wrote: Hi again, On the other hand, we are running the following flink CLI command: ./flink run -d -m ${jobmanager.rpc.address}:${jobmanager.rpc.port} ${our-program-jar} ${our-pr

Re: Flink CLI cannot submit job to Flink on Mesos

2017-07-31 Thread Francisco Gonzalez Barea
leader session id) from the HA store (ZooKeeper). [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/mesos.html#high-availability Cheers, Till On Thu, Jul 27, 2017 at 6:20 PM, Francisco Gonzalez Barea mailto:francisco.gonza...@piksel.com>> wrote: Hello, We´re having l

Re: Flink CLI cannot submit job to Flink on Mesos

2017-07-31 Thread Francisco Gonzalez Barea
Keeper). [1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/mesos.html#high-availability Cheers, Till On Thu, Jul 27, 2017 at 6:20 PM, Francisco Gonzalez Barea mailto:francisco.gonza...@piksel.com>> wrote: Hello, We´re having lot of issues while trying to submit a job re

Flink CLI cannot submit job to Flink on Mesos

2017-07-27 Thread Francisco Gonzalez Barea
Hello, We´re having lot of issues while trying to submit a job remotely using the Flink CLI command line tool. We have tried different configurations but in all of them we get errors from AKKA while trying to connect. I will try to summarise the configurations we´ve tried. - Flink 1.3.0 deploy