Re: How do I start a Flink application on my Flink+Mesos cluster?

[Gary Yao]

Hi Felipe,

I am glad that you were able to fix the problem yourself.

> But I suppose that Mesos will allocate Slots and Task Managers
dynamically.
> Is that right?

Yes, that is the case since Flink 1.5 [1].

> Then I put the number of "mesos.resourcemanager.tasks.cpus" to be equal or
> less the available cores on a single node of the cluster. I am not sure
about
> this parameter, but only after this configuration it worked.

I would need to see JobManager and Mesos logs to understand why this
resolved
your issue. If you do not set mesos.resourcemanager.tasks.cpus explicitly,
Flink will request CPU resources equal to the number of TaskManager slots
(taskmanager.numberOfTaskSlots) [2]. Maybe this value was too high in your
configuration?

Best,
Gary


[1]
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=65147077
[2]
https://github.com/apache/flink/blob/0a405251b297109fde1f9a155eff14be4d943887/flink-mesos/src/main/java/org/apache/flink/mesos/runtime/clusterframework/MesosTaskManagerParameters.java#L344

On Tue, Sep 10, 2019 at 10:41 AM Felipe Gutierrez <
felipe.o.gutier...@gmail.com> wrote:

> I managed to find what was going wrong. I will write here just for the
> record.
>
> First, the master machine was not login automatically at itself. So I had
> to give permission for it.
>
> chmod og-wx ~/.ssh/authorized_keys
> chmod 750 $HOME
>
> Then I put the number of "mesos.resourcemanager.tasks.cpus" to be equal or
> less the available cores on a single node of the cluster. I am not sure
> about this parameter, but only after this configuration it worked.
>
> Felipe
> *--*
> *-- Felipe Gutierrez*
>
> *-- skype: felipe.o.gutierrez*
> *--* *https://felipeogutierrez.blogspot.com
> *
>
>
> On Fri, Sep 6, 2019 at 10:36 AM Felipe Gutierrez <
> felipe.o.gutier...@gmail.com> wrote:
>
>> Hi,
>>
>> I am running Mesos without DC/OS [1] and Flink on it. Whe I start my
>> cluster I receive some messages that I suppose everything was started.
>> However, I see 0 slats available on the Flink web dashboard. But I suppose
>> that Mesos will allocate Slots and Task Managers dynamically. Is that right?
>>
>> $ ./bin/mesos-appmaster.sh &
>> [1] 16723
>> flink@r03:~/flink-1.9.0$ I0906 10:22:45.080328 16943 sched.cpp:239]
>> Version: 1.9.0
>> I0906 10:22:45.082672 16996 sched.cpp:343] New master detected at
>> mas...@xxx.xxx.xxx.xxx:5050
>> I0906 10:22:45.083276 16996 sched.cpp:363] No credentials provided.
>> Attempting to register without authentication
>> I0906 10:22:45.086840 16997 sched.cpp:751] Framework registered with
>> 22f6a553-e8ac-42d4-9a90-96a8d5f002f0-0003
>>
>> Then I deploy my Flink application. When I use the first command to
>> deploy the application starts. However, the tasks remain CREATED until
>> Flink throws a timeout exception. In other words, it never turns to RUNNING.
>> When I use the second comman to deploy the application it does not start
>> and I receive the exception of "Could not allocate all requires slots
>> within timeout of 30 ms. Slots required: 2". The full stacktrace is
>> below.
>>
>> $ /home/flink/flink-1.9.0/bin/flink run
>> /home/flink/hello-flink-mesos/target/hello-flink-mesos.jar &
>> $ ./bin/mesos-appmaster-job.sh run
>> /home/flink/hello-flink-mesos/target/hello-flink-mesos.jar &
>>
>>
>> [1]
>> https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/mesos.html#mesos-without-dcos
>> ps.: my application runs normally on a standalone Flink cluster.
>>
>> 
>>  The program finished with the following exception:
>>
>> org.apache.flink.client.program.ProgramInvocationException: Job failed.
>> (JobID: 7ad8d71faaceb1ac469353452c43dc2a)
>> at
>> org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:262)
>> at
>> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:338)
>> at
>> org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:60)
>> at
>> org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1507)
>> at org.hello_flink_mesos.App.(App.java:35)
>> at org.hello_flink_mesos.App.main(App.java:285)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at
>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576)
>> at
>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438)
>> at
>> org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:274)
>> at
>> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:746)
>> at
>> org.apache.flink.client.cli.

Re: error in Elasticsearch

[Gary Yao]

Hi,

You are not supposed to change that part of the exercise code. You have to
pass the path to the input file as a program argument (e.g., --input
/path/to/file). See [1] and [2] on how to configure program arguments in
IntelliJ.

Best,
Gary

[1]
https://www.jetbrains.com/help/idea/run-debug-configuration-application.html#1
[2]
https://stackoverflow.com/questions/2066307/how-do-you-input-commandline-argument-in-intellij-idea

On Fri, Sep 6, 2019 at 1:41 PM alaa  wrote:

> *Hallo
>
> I try to implement this example to write the results of Popular Places into
> an Elasticsearch index.
>
> But when I run a code .. there was some Error appear *
>
>
> <
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t1965/Screenshot_from_2019-09-06_13-29-15.png>
>
>
>
> *
> and when i set the path-to-input-file .. also there was an error appear
>
> <
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t1965/Screenshot_from_2019-09-06_13-36-10.png>
>
> *
>
> *Can you help me which parameter should i put in this Line
>
> String input = params.getRequired("input");*
>
>
> Thank you
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>

Re: Checkpointing is not performing well

[Ravi Bhushan Ratnakar]

What is the upper limit of checkpoint size of Flink System?

Regards,
Ravi

On Wed 11 Sep, 2019, 06:48 Vijay Bhaskar,  wrote:

> You crossed  the upper limits of the check point system of Flink a way
> high. Try to distribute events equally over time by adding some sort of
> controlled back pressure after receiving data from kinesis streams.
> Otherwise the spike coming during 5 seconds time would always create
> problems. Tomorrow it may double so best solution in your case is to
> deliver at configurable constant rate after receiving messages from kinesis
> streams. Otherwise i am sure its always the problem whatever the kind of
> streaming engine you use. Tune your configuration to get the optimal rate
> so that flink checkpoint state is healthier.
>
> Regards
> Bhaskar
>
> On Tue, Sep 10, 2019 at 11:16 PM Ravi Bhushan Ratnakar <
> ravibhushanratna...@gmail.com> wrote:
>
>> @Rohan - I am streaming data to kafka sink after applying business logic.
>> For checkpoint, I am using s3 as a distributed file system. For local
>> recovery, I am using Optimized iops ebs volume.
>>
>> @Vijay - I forget to mention that incoming data volume is ~ 10 to 21GB
>> per minute compressed(lz4) avro message. Generally 90% correlated events
>> come within 5 seconds and 10% of the correlated events get extended to 65
>> minute. Due to this business requirement, the state size keep growing till
>> 65 minutes, after that the state size becomes more or less stable. As the
>> state size is growing and is around 350gb at peak load, checkpoint is not
>> able to complete within 1 minutes. I want to check as quick as possible
>> like every 5 second.
>>
>> Thanks,
>> Ravi
>>
>>
>> On Tue 10 Sep, 2019, 11:37 Vijay Bhaskar, 
>> wrote:
>>
>>> For me task count seems to be huge in number with the mentioned resource
>>> count. To rule out the possibility of issue with state backend can you
>>> start writing sink data as  , i.e., data ignore sink. And try
>>> whether you could run it for longer duration without any issue. You can
>>> start decreasing the task manager count until you find descent count of it
>>> without having any side effects. Use that value as task manager count and
>>> then start adding your state backend. First you can try with Rocks DB. With
>>> reduced task manager count you might get good results.
>>>
>>> Regards
>>> Bhaskar
>>>
>>> On Sun, Sep 8, 2019 at 10:15 AM Rohan Thimmappa <
>>> rohan.thimma...@gmail.com> wrote:
>>>
 Ravi, have you looked at the io operation(iops) rate of the disk? You
 can monitoring the iops performance and tune it accordingly with your work
 load. This helped us in our project when we hit the wall tuning prototype
 much all the parameters.

 Rohan


 --
 *From:* Ravi Bhushan Ratnakar 
 *Sent:* Saturday, September 7, 2019 5:38 PM
 *To:* Rafi Aroch
 *Cc:* user
 *Subject:* Re: Checkpointing is not performing well

 Hi Rafi,

 Thank you for your quick response.

 I have tested with rocksdb state backend. Rocksdb required
 significantly more taskmanager to perform as compare to filesystem state
 backend. The problem here is that checkpoint process is not fast enough to
 complete.

 Our requirement is to do checkout as soon as possible like in 5 seconds
 to flush the output to output sink. As the incoming data rate is high, it
 is not able to complete quickly. If I increase the checkpoint duration, the
 state size grows much faster and hence takes much longer time to complete
 checkpointing. I also tried to use AT LEAST ONCE mode, but does not improve
 much. Adding more taskmanager to increase parallelism also does not improve
 the checkpointing performance.

 Is it possible to achieve checkpointing as short as 5 seconds with such
 high input volume?

 Regards,
 Ravi

 On Sat 7 Sep, 2019, 22:25 Rafi Aroch,  wrote:

> Hi Ravi,
>
> Consider moving to RocksDB state backend, where you can enable
> incremental checkpointing. This will make you checkpoints size stay pretty
> much constant even when your state becomes larger.
>
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/state/state_backends.html#the-rocksdbstatebackend
>
>
> Thanks,
> Rafi
>
> On Sat, Sep 7, 2019, 17:47 Ravi Bhushan Ratnakar <
> ravibhushanratna...@gmail.com> wrote:
>
>> Hi All,
>>
>> I am writing a streaming application using Flink 1.9. This
>> application consumes data from kinesis stream which is basically avro
>> payload. Application is using KeyedProcessFunction to execute business
>> logic on the basis of correlation id using event time characteristics 
>> with
>> below configuration --
>> StateBackend - filesystem with S3 storage
>> registerTimeTimer duration for each key is  -  currentWatermark  + 15
>> seconds

Re: Filter events based on future events

[Fabian Hueske]

Hi Theo,

I would implement this with a KeyedProcessFunction.
These are the important points to consider:

1) partition the output of the Kafka source by Kafka partition (or the
attribute that determines the partition). This will ensure that the data
stay in order (per partition).
2) The KeyedProcessFunction needs state to buffer the data of one minute.
It depends on the amount of data that you expect to buffer which state is
the most efficient. If you expect that one minute can be easily hold in
memory, I'd use a FS state backend which keeps all state on the JVM heap.
You could use a ValueState with an appropriate data structure (Queue,
PrioQueue, ...). The data structure would be held as regular Java object on
the heap and hence provide efficient access. If you expect the one minute
to be too much data to be held in memory, you need to go for the RocksDB
state backend. Since this causes de/serialization with every read and write
access, it's more difficult to identify an efficient state primitive /
access pattern. I won't go into the details here, assuming that the
buffered data fits into memory and you can go for the FS state backend. If
that's not the case, let me know and I can share some tips on the RocksDB
state backend approach. The KeyedProcessFunction would add records to the
buffer state when processElement() is called and emit all buffered records
that have a timestamp of less than the timestamp of the currently added
record - 1 minute.

Note, since the timestamps are monotonically increasing, we do not need
watermarks and event-time but can rely on the timestamps of the records.
Hence, the application won't block if one partition stalls providing the
same benefits that per-key watermarks would offer (if they were supported
by Flink).

Best, Fabian

Am Di., 10. Sept. 2019 um 23:06 Uhr schrieb
theo.diefent...@scoop-software.de :

> Hi there,
>
> I have the following use case:
>
> I get transaction logs from multiple servers. Each server puts its logs
> into its own Kafka partition so that within each partition the elements are
> monothonically ordered by time.
>
> Within the stream of transactions, we have some special events. Let's call
> them A. (roughly 1-10% in distribution have this type).
>
> An A event can have an Anti-A event later on in time. That is an event
> which has all the same attributes (like username, faculty,..) but differs
> in one boolean attribute indicating that it is an anti event. Kind of a
> retraction.
>
> Now I want to emit almost all events downstream (including neither A nor
> Anti-A, let's call them simpy B), preserving the monothonical order of
> events. There is just one special case in which I want to filter out an
> element: If the stream has an A event followed by an Anti-A event within
> one minute time, only the Anti-A event shall go downstream, not A itself.
> But if there is no Anti-A event, A shall be emitted and shall still be
> within timestamp order of events.
>
> I'm wrangling my head around it a lot and don't come up with a proper
> (performant) solution. It seems to be obvious that in the end, I need to
> buffer all records over 1 minute so that order can be preserved. But I have
> no idea how to implement this in Flink efficiently.
>
> My thoughts thus far:
>
> 1. I could give CEP a try. But in that CEP I would need to write something
> like match all B events in any case. And match A also but only if there is
> no anti A => doesn`t that produce a lot of state? And are all B events
> considered in the breadth first rule match approach, I. E. Tons of
> unnecessary comparisons against A? Any pseudo code on how I could do this
> with CEP?
>
> 2. If I key data by partition and all other attributes except for the
> retract boolean so that A and anti A always fall into the same keyed stream
> but no other event in that stream, I probably get much better comparison
> capabilities. But how much overhead do I produce with it? Will Flink
> reshuffle the data even if the first key stays the same? And can I
> backpartiton to my "global" per partition order? Note that some events have
> the exact event time timestamp but I still want to have them in their
> original order later on.
>
> 3. Could I work with session windows somehow? Putting A and Anti A in the
> same session and in window emit I would just not collect the A event if
> there is an Anti A? Would it be more or less overhead compared to CEP?
>
> 4. Do you have any other idea on how to approach this? Sadly, I have no
> way to manipulate the input stream, so that part of the pipeline is fixed.
>
> Best regards
> Theo
>

Re:

[Fabian Hueske]

Hi,

This is clearly a Scala version issue.
You need to make sure that all Flink dependencies have the same version and
are compiled for Scala 2.11.
The "_2.11" postfix in the dependency name indicates that it is a Scala
2.11 dependency ("_2.12 indicates Scala 2.12 compatibility).

Best, Fabian

Am Mi., 11. Sept. 2019 um 05:53 Uhr schrieb Ben Yan <
yan.xiao.bin.m...@gmail.com>:

> The following is the environment I use：
> 1. flink.version: 1.9.0
> 2. java version "1.8.0_212"
> 3. scala version: 2.11.12
>
> When I wrote the following code in the scala programming language, I found
> the following error：
>
> // set up the batch execution environment
> val bbSettings = 
> EnvironmentSettings.newInstance().useBlinkPlanner().inBatchMode().build()
> val bbTableEnv = TableEnvironment.create(bbSettings)
>
> error: Static methods in interface require -target:jvm-1.8
> [ERROR] val bbTableEnv = TableEnvironment.create(bbSettings)
>
> But when I use the java programming language or the version of scala in 2.12, 
> there is no problem.
>
> If I use the version of scala2.11, is there any way to solve this problem? 
> thanks
>
>
> Best,
>
> Ben
>
>

Re: Checkpointing is not performing well

[Fabian Hueske]

Hi,

There is no upper limit for state size in Flink. There are applications
with 10+ TB state.
However, it is natural that checkpointing time increases with state size as
more data needs to be serialized (in case of FSStateBackend) and written to
stable storage.
(The same is btw true for recovery when the state needs to be loaded back.)

There are a few tricks to reduce checkpointing time like using incremental
checkpoints which you tried already.
You can also scale out the application to use more machines and therefore
bandwidth + CPU (for serialization) during checkpoints.

Fabian

Am Mi., 11. Sept. 2019 um 09:38 Uhr schrieb Ravi Bhushan Ratnakar <
ravibhushanratna...@gmail.com>:

> What is the upper limit of checkpoint size of Flink System?
>
> Regards,
> Ravi
>
> On Wed 11 Sep, 2019, 06:48 Vijay Bhaskar, 
> wrote:
>
>> You crossed  the upper limits of the check point system of Flink a way
>> high. Try to distribute events equally over time by adding some sort of
>> controlled back pressure after receiving data from kinesis streams.
>> Otherwise the spike coming during 5 seconds time would always create
>> problems. Tomorrow it may double so best solution in your case is to
>> deliver at configurable constant rate after receiving messages from kinesis
>> streams. Otherwise i am sure its always the problem whatever the kind of
>> streaming engine you use. Tune your configuration to get the optimal rate
>> so that flink checkpoint state is healthier.
>>
>> Regards
>> Bhaskar
>>
>> On Tue, Sep 10, 2019 at 11:16 PM Ravi Bhushan Ratnakar <
>> ravibhushanratna...@gmail.com> wrote:
>>
>>> @Rohan - I am streaming data to kafka sink after applying business
>>> logic. For checkpoint, I am using s3 as a distributed file system. For
>>> local recovery, I am using Optimized iops ebs volume.
>>>
>>> @Vijay - I forget to mention that incoming data volume is ~ 10 to 21GB
>>> per minute compressed(lz4) avro message. Generally 90% correlated events
>>> come within 5 seconds and 10% of the correlated events get extended to 65
>>> minute. Due to this business requirement, the state size keep growing till
>>> 65 minutes, after that the state size becomes more or less stable. As the
>>> state size is growing and is around 350gb at peak load, checkpoint is not
>>> able to complete within 1 minutes. I want to check as quick as possible
>>> like every 5 second.
>>>
>>> Thanks,
>>> Ravi
>>>
>>>
>>> On Tue 10 Sep, 2019, 11:37 Vijay Bhaskar, 
>>> wrote:
>>>
 For me task count seems to be huge in number with the mentioned
 resource count. To rule out the possibility of issue with state backend can
 you start writing sink data as  , i.e., data ignore sink. And
 try whether you could run it for longer duration without any issue. You can
 start decreasing the task manager count until you find descent count of it
 without having any side effects. Use that value as task manager count and
 then start adding your state backend. First you can try with Rocks DB. With
 reduced task manager count you might get good results.

 Regards
 Bhaskar

 On Sun, Sep 8, 2019 at 10:15 AM Rohan Thimmappa <
 rohan.thimma...@gmail.com> wrote:

> Ravi, have you looked at the io operation(iops) rate of the disk? You
> can monitoring the iops performance and tune it accordingly with your work
> load. This helped us in our project when we hit the wall tuning prototype
> much all the parameters.
>
> Rohan
>
>
> --
> *From:* Ravi Bhushan Ratnakar 
> *Sent:* Saturday, September 7, 2019 5:38 PM
> *To:* Rafi Aroch
> *Cc:* user
> *Subject:* Re: Checkpointing is not performing well
>
> Hi Rafi,
>
> Thank you for your quick response.
>
> I have tested with rocksdb state backend. Rocksdb required
> significantly more taskmanager to perform as compare to filesystem state
> backend. The problem here is that checkpoint process is not fast enough to
> complete.
>
> Our requirement is to do checkout as soon as possible like in 5
> seconds to flush the output to output sink. As the incoming data rate is
> high, it is not able to complete quickly. If I increase the checkpoint
> duration, the state size grows much faster and hence takes much longer 
> time
> to complete checkpointing. I also tried to use AT LEAST ONCE mode, but 
> does
> not improve much. Adding more taskmanager to increase parallelism also 
> does
> not improve the checkpointing performance.
>
> Is it possible to achieve checkpointing as short as 5 seconds with
> such high input volume?
>
> Regards,
> Ravi
>
> On Sat 7 Sep, 2019, 22:25 Rafi Aroch,  wrote:
>
>> Hi Ravi,
>>
>> Consider moving to RocksDB state backend, where you can enable
>> incremental checkpointing. This will make you checkpoints size stay 
>> pretty
>> much const

[DISCUSS] Drop older versions of Kafka Connectors (0.9, 0.10) for Flink 1.10

[Stephan Ewen]

Hi all!

We still maintain connectors for Kafka 0.8 and 0.9 in Flink.
I would suggest to drop those with Flink 1.10 and start supporting only
Kafka 0.10 onwards.

Are there any concerns about this, or still a significant number of users
of these versions?

Best,
Stephan

Re: [DISCUSS] Drop older versions of Kafka Connectors (0.9, 0.10) for Flink 1.10

[Wesley Peng]

on 2019/9/11 16:17, Stephan Ewen wrote:

We still maintain connectors for Kafka 0.8 and 0.9 in Flink.
I would suggest to drop those with Flink 1.10 and start supporting only 
Kafka 0.10 onwards.


Are there any concerns about this, or still a significant number of 
users of these versions?


But we still have a large scale of deployment kafka 0.9 in production. 
Do you mean the new coming flink won't support kafka 0.9?

Though I understand for it, but just sounds sorry. :)

regards.

Re: error in Elasticsearch

[alaa]

 Hallo 

I put arguments but the same error appear .. what should i do ?

 

 



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: suggestion of FLINK-10868

[Anyang Hu]

Hi Till,

Some of our online batch tasks have strict SLA requirements, and they are
not allowed to be stuck for a long time. Therefore, we take a rude way to
make the job exit immediately. The way to wait for connection recovery is a
better solution. Maybe we need to add a timeout to wait for JM to restore
the connection?

For suggestion 1, make interval configurable, given that we have done it,
and if we can, we hope to give back to the community.

Best regards,
Anyang

Till Rohrmann  于2019年9月9日周一 下午3:09写道：

> Hi Anyang,
>
> I think we cannot take your proposal because this means that whenever we
> want to call notifyAllocationFailure when there is a connection problem
> between the RM and the JM, then we fail the whole cluster. This is
> something a robust and resilient system should not do because connection
> problems are expected and need to be handled gracefully. Instead if one
> deems the notifyAllocationFailure message to be very important, then one
> would need to keep it and tell the JM once it has connected back.
>
> Cheers,
> Till
>
> On Sun, Sep 8, 2019 at 11:26 AM Anyang Hu  wrote:
>
>> Hi Peter,
>>
>> For our online batch task, there is a scene where the failed Container
>> reaches MAXIMUM_WORKERS_FAILURE_RATE but the client will not immediately
>> exit (the probability of JM loss is greatly improved when thousands of
>> Containers is to be started). It is found that the JM disconnection (the
>> reason for JM loss is unknown) will cause the notifyAllocationFailure not
>> to take effect.
>>
>> After the introduction of FLINK-13184
>>  to start  the
>> container with multi-threaded, the JM disconnection situation has been
>> alleviated. In order to stably implement the client immediate exit, we use
>> the following code to determine  whether call onFatalError when
>> MaximumFailedTaskManagerExceedingException is occurd:
>>
>> @Override
>> public void notifyAllocationFailure(JobID jobId, AllocationID allocationId, 
>> Exception cause) {
>>validateRunsInMainThread();
>>
>>JobManagerRegistration jobManagerRegistration = 
>> jobManagerRegistrations.get(jobId);
>>if (jobManagerRegistration != null) {
>>   
>> jobManagerRegistration.getJobManagerGateway().notifyAllocationFailure(allocationId,
>>  cause);
>>} else {
>>   if (exitProcessOnJobManagerTimedout) {
>>  ResourceManagerException exception = new 
>> ResourceManagerException("Job Manager is lost, can not notify allocation 
>> failure.");
>>  onFatalError(exception);
>>   }
>>}
>> }
>>
>>
>> Best regards,
>>
>> Anyang
>>
>>

Re: suggestion of FLINK-10868

[Till Rohrmann]

Suggestion 1 makes sense. For the quick termination I think we need to
think a bit more about it to find a good solution also to support strict
SLA requirements.

Cheers,
Till

On Wed, Sep 11, 2019 at 11:11 AM Anyang Hu  wrote:

> Hi Till,
>
> Some of our online batch tasks have strict SLA requirements, and they are
> not allowed to be stuck for a long time. Therefore, we take a rude way to
> make the job exit immediately. The way to wait for connection recovery is a
> better solution. Maybe we need to add a timeout to wait for JM to restore
> the connection?
>
> For suggestion 1, make interval configurable, given that we have done it,
> and if we can, we hope to give back to the community.
>
> Best regards,
> Anyang
>
> Till Rohrmann  于2019年9月9日周一 下午3:09写道：
>
>> Hi Anyang,
>>
>> I think we cannot take your proposal because this means that whenever we
>> want to call notifyAllocationFailure when there is a connection problem
>> between the RM and the JM, then we fail the whole cluster. This is
>> something a robust and resilient system should not do because connection
>> problems are expected and need to be handled gracefully. Instead if one
>> deems the notifyAllocationFailure message to be very important, then one
>> would need to keep it and tell the JM once it has connected back.
>>
>> Cheers,
>> Till
>>
>> On Sun, Sep 8, 2019 at 11:26 AM Anyang Hu  wrote:
>>
>>> Hi Peter,
>>>
>>> For our online batch task, there is a scene where the failed Container
>>> reaches MAXIMUM_WORKERS_FAILURE_RATE but the client will not immediately
>>> exit (the probability of JM loss is greatly improved when thousands of
>>> Containers is to be started). It is found that the JM disconnection (the
>>> reason for JM loss is unknown) will cause the notifyAllocationFailure not
>>> to take effect.
>>>
>>> After the introduction of FLINK-13184
>>>  to start  the
>>> container with multi-threaded, the JM disconnection situation has been
>>> alleviated. In order to stably implement the client immediate exit, we use
>>> the following code to determine  whether call onFatalError when
>>> MaximumFailedTaskManagerExceedingException is occurd:
>>>
>>> @Override
>>> public void notifyAllocationFailure(JobID jobId, AllocationID allocationId, 
>>> Exception cause) {
>>>validateRunsInMainThread();
>>>
>>>JobManagerRegistration jobManagerRegistration = 
>>> jobManagerRegistrations.get(jobId);
>>>if (jobManagerRegistration != null) {
>>>   
>>> jobManagerRegistration.getJobManagerGateway().notifyAllocationFailure(allocationId,
>>>  cause);
>>>} else {
>>>   if (exitProcessOnJobManagerTimedout) {
>>>  ResourceManagerException exception = new 
>>> ResourceManagerException("Job Manager is lost, can not notify allocation 
>>> failure.");
>>>  onFatalError(exception);
>>>   }
>>>}
>>> }
>>>
>>>
>>> Best regards,
>>>
>>> Anyang
>>>
>>>

[ANNOUNCE] Zili Chen becomes a Flink committer

[Till Rohrmann]

Hi everyone,

I'm very happy to announce that Zili Chen (some of you might also know him
as Tison Kun) accepted the offer of the Flink PMC to become a committer of
the Flink project.

Zili Chen has been an active community member for almost 16 months now. He
helped pushing the Flip-6 effort over the finish line, ported a lot of
legacy code tests, removed a good part of the legacy code, contributed
numerous fixes, is involved in the Flink's client API refactoring, drives
the refactoring of Flink's HighAvailabilityServices and much more. Zili
Chen also helped the community by PR reviews, reporting Flink issues,
answering user mails and being very active on the dev mailing list.

Congratulations Zili Chen!

Best, Till
(on behalf of the Flink PMC)

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Wesley Peng]

Hi

on 2019/9/11 17:22, Till Rohrmann wrote:
I'm very happy to announce that Zili Chen (some of you might also know 
him as Tison Kun) accepted the offer of the Flink PMC to become a 
committer of the Flink project.


Congratulations Zili Chen.

regards.

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Jeff Zhang]

Congratulations Zili Chen!

Wesley Peng  于2019年9月11日周三 下午5:25写道：

> Hi
>
> on 2019/9/11 17:22, Till Rohrmann wrote:
> > I'm very happy to announce that Zili Chen (some of you might also know
> > him as Tison Kun) accepted the offer of the Flink PMC to become a
> > committer of the Flink project.
>
> Congratulations Zili Chen.
>
> regards.
>


-- 
Best Regards

Jeff Zhang

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Dian Fu]

Congratulations!

> 在 2019年9月11日，下午5:26，Jeff Zhang  写道：
> 
> Congratulations Zili Chen! 
> 
> Wesley Peng mailto:wes...@thepeng.eu>> 于2019年9月11日周三 
> 下午5:25写道：
> Hi
> 
> on 2019/9/11 17:22, Till Rohrmann wrote:
> > I'm very happy to announce that Zili Chen (some of you might also know 
> > him as Tison Kun) accepted the offer of the Flink PMC to become a 
> > committer of the Flink project.
> 
> Congratulations Zili Chen.
> 
> regards.
> 
> 
> -- 
> Best Regards
> 
> Jeff Zhang

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Terry Wang]

Congratulations!

Best,
Terry Wang



> 在 2019年9月11日，下午5:28，Dian Fu  写道：
> 
> Congratulations!
> 
>> 在 2019年9月11日，下午5:26，Jeff Zhang mailto:zjf...@gmail.com>> 
>> 写道：
>> 
>> Congratulations Zili Chen! 
>> 
>> Wesley Peng mailto:wes...@thepeng.eu>> 于2019年9月11日周三 
>> 下午5:25写道：
>> Hi
>> 
>> on 2019/9/11 17:22, Till Rohrmann wrote:
>> > I'm very happy to announce that Zili Chen (some of you might also know 
>> > him as Tison Kun) accepted the offer of the Flink PMC to become a 
>> > committer of the Flink project.
>> 
>> Congratulations Zili Chen.
>> 
>> regards.
>> 
>> 
>> -- 
>> Best Regards
>> 
>> Jeff Zhang
> 

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Zhu Zhu]

Congratulations Zili!

Thanks,
Zhu Zhu

Terry Wang  于2019年9月11日周三 下午5:34写道：

> Congratulations!
>
> Best,
> Terry Wang
>
>
>
> 在 2019年9月11日，下午5:28，Dian Fu  写道：
>
> Congratulations!
>
> 在 2019年9月11日，下午5:26，Jeff Zhang  写道：
>
> Congratulations Zili Chen!
>
> Wesley Peng  于2019年9月11日周三 下午5:25写道：
>
>> Hi
>>
>> on 2019/9/11 17:22, Till Rohrmann wrote:
>> > I'm very happy to announce that Zili Chen (some of you might also know
>> > him as Tison Kun) accepted the offer of the Flink PMC to become a
>> > committer of the Flink project.
>>
>> Congratulations Zili Chen.
>>
>> regards.
>>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
>
>
>

Re: error in Elasticsearch

[Gary Yao]

Program arguments should be set to "--input /home/alaa/nycTaxiRides.gz"
(without the quotes).

On Wed, Sep 11, 2019 at 10:39 AM alaa  wrote:

>  Hallo
>
> I put arguments but the same error appear .. what should i do ?
>
>
> <
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/file/t1965/Screenshot_from_2019-09-11_10-34-42.png>
>
>
>
>
> --
> Sent from:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[bupt_ljy]

Congratulations!


Best,
Jiayi Liao


 Original Message 
Sender: Till Rohrmann
Recipient: dev; user
Date: Wednesday, Sep 11, 2019 17:22
Subject: [ANNOUNCE] Zili Chen becomes a Flink committer


Hi everyone,


I'm very happy to announce that Zili Chen (some of you might also know him as 
Tison Kun) accepted the offer of the Flink PMC to become a committer of the 
Flink project.


Zili Chen has been an active community member for almost 16 months now. He 
helped pushing the Flip-6 effort over the finish line, ported a lot of legacy 
code tests, removed a good part of the legacy code, contributed numerous fixes, 
is involved in the Flink's client API refactoring, drives the refactoring of 
Flink's HighAvailabilityServices and much more. Zili Chen also helped the 
community by PR reviews, reporting Flink issues, answering user mails and being 
very active on the dev mailing list.


Congratulations Zili Chen!


Best, Till 

(on behalf of the Flink PMC)

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Oytun Tez]

Congratulations!

---
Oytun Tez

*M O T A W O R D*
The World's Fastest Human Translation Platform.
oy...@motaword.com — www.motaword.com


On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:

> Congratulations!
>
>
> Best,
>
> Jiayi Liao
>
>  Original Message
> *Sender:* Till Rohrmann
> *Recipient:* dev; user
> *Date:* Wednesday, Sep 11, 2019 17:22
> *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
>
> Hi everyone,
>
> I'm very happy to announce that Zili Chen (some of you might also know
> him as Tison Kun) accepted the offer of the Flink PMC to become a committer
> of the Flink project.
>
> Zili Chen has been an active community member for almost 16 months now.
> He helped pushing the Flip-6 effort over the finish line, ported a lot of
> legacy code tests, removed a good part of the legacy code, contributed
> numerous fixes, is involved in the Flink's client API refactoring, drives
> the refactoring of Flink's HighAvailabilityServices and much more. Zili
> Chen also helped the community by PR reviews, reporting Flink issues,
> answering user mails and being very active on the dev mailing list.
>
> Congratulations Zili Chen!
>
> Best, Till
> (on behalf of the Flink PMC)
>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Benchao Li]

Congratulations Zili Chen!

bupt_ljy  于2019年9月11日周三 下午6:36写道：

> Congratulations!
>
>
> Best,
>
> Jiayi Liao
>
>  Original Message
> *Sender:* Till Rohrmann
> *Recipient:* dev; user
> *Date:* Wednesday, Sep 11, 2019 17:22
> *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
>
> Hi everyone,
>
> I'm very happy to announce that Zili Chen (some of you might also know
> him as Tison Kun) accepted the offer of the Flink PMC to become a committer
> of the Flink project.
>
> Zili Chen has been an active community member for almost 16 months now.
> He helped pushing the Flip-6 effort over the finish line, ported a lot of
> legacy code tests, removed a good part of the legacy code, contributed
> numerous fixes, is involved in the Flink's client API refactoring, drives
> the refactoring of Flink's HighAvailabilityServices and much more. Zili
> Chen also helped the community by PR reviews, reporting Flink issues,
> answering user mails and being very active on the dev mailing list.
>
> Congratulations Zili Chen!
>
> Best, Till
> (on behalf of the Flink PMC)
>


-- 

Benchao Li
School of Electronics Engineering and Computer Science, Peking University
Tel:+86-15650713730
Email: libenc...@gmail.com; libenc...@pku.edu.cn

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Biao Liu]

Congrats Zili!

Thanks,
Biao /'bɪ.aʊ/



On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:

> Congratulations!
>
> ---
> Oytun Tez
>
> *M O T A W O R D*
> The World's Fastest Human Translation Platform.
> oy...@motaword.com — www.motaword.com
>
>
> On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:
>
>> Congratulations!
>>
>>
>> Best,
>>
>> Jiayi Liao
>>
>>  Original Message
>> *Sender:* Till Rohrmann
>> *Recipient:* dev; user
>> *Date:* Wednesday, Sep 11, 2019 17:22
>> *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
>>
>> Hi everyone,
>>
>> I'm very happy to announce that Zili Chen (some of you might also know
>> him as Tison Kun) accepted the offer of the Flink PMC to become a committer
>> of the Flink project.
>>
>> Zili Chen has been an active community member for almost 16 months now.
>> He helped pushing the Flip-6 effort over the finish line, ported a lot of
>> legacy code tests, removed a good part of the legacy code, contributed
>> numerous fixes, is involved in the Flink's client API refactoring, drives
>> the refactoring of Flink's HighAvailabilityServices and much more. Zili
>> Chen also helped the community by PR reviews, reporting Flink issues,
>> answering user mails and being very active on the dev mailing list.
>>
>> Congratulations Zili Chen!
>>
>> Best, Till
>> (on behalf of the Flink PMC)
>>
>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Fabian Hueske]

Congrats Zili Chen :-)

Cheers, Fabian

Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu :

> Congrats Zili!
>
> Thanks,
> Biao /'bɪ.aʊ/
>
>
>
> On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:
>
>> Congratulations!
>>
>> ---
>> Oytun Tez
>>
>> *M O T A W O R D*
>> The World's Fastest Human Translation Platform.
>> oy...@motaword.com — www.motaword.com
>>
>>
>> On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:
>>
>>> Congratulations!
>>>
>>>
>>> Best,
>>>
>>> Jiayi Liao
>>>
>>>  Original Message
>>> *Sender:* Till Rohrmann
>>> *Recipient:* dev; user
>>> *Date:* Wednesday, Sep 11, 2019 17:22
>>> *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
>>>
>>> Hi everyone,
>>>
>>> I'm very happy to announce that Zili Chen (some of you might also know
>>> him as Tison Kun) accepted the offer of the Flink PMC to become a committer
>>> of the Flink project.
>>>
>>> Zili Chen has been an active community member for almost 16 months now.
>>> He helped pushing the Flip-6 effort over the finish line, ported a lot of
>>> legacy code tests, removed a good part of the legacy code, contributed
>>> numerous fixes, is involved in the Flink's client API refactoring, drives
>>> the refactoring of Flink's HighAvailabilityServices and much more. Zili
>>> Chen also helped the community by PR reviews, reporting Flink issues,
>>> answering user mails and being very active on the dev mailing list.
>>>
>>> Congratulations Zili Chen!
>>>
>>> Best, Till
>>> (on behalf of the Flink PMC)
>>>
>>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Guowei Ma]

Congratulations Zili !

Best,
Guowei


Fabian Hueske  于2019年9月11日周三 下午7:02写道：

> Congrats Zili Chen :-)
>
> Cheers, Fabian
>
> Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu :
>
>> Congrats Zili!
>>
>> Thanks,
>> Biao /'bɪ.aʊ/
>>
>>
>>
>> On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:
>>
>>> Congratulations!
>>>
>>> ---
>>> Oytun Tez
>>>
>>> *M O T A W O R D*
>>> The World's Fastest Human Translation Platform.
>>> oy...@motaword.com — www.motaword.com
>>>
>>>
>>> On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:
>>>
 Congratulations!


 Best,

 Jiayi Liao

  Original Message
 *Sender:* Till Rohrmann
 *Recipient:* dev; user
 *Date:* Wednesday, Sep 11, 2019 17:22
 *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer

 Hi everyone,

 I'm very happy to announce that Zili Chen (some of you might also know
 him as Tison Kun) accepted the offer of the Flink PMC to become a committer
 of the Flink project.

 Zili Chen has been an active community member for almost 16 months
 now. He helped pushing the Flip-6 effort over the finish line, ported a lot
 of legacy code tests, removed a good part of the legacy code, contributed
 numerous fixes, is involved in the Flink's client API refactoring, drives
 the refactoring of Flink's HighAvailabilityServices and much more. Zili
 Chen also helped the community by PR reviews, reporting Flink issues,
 answering user mails and being very active on the dev mailing list.

 Congratulations Zili Chen!

 Best, Till
 (on behalf of the Flink PMC)

>>>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[SHI Xiaogang]

Congratulations!

Regards,
Xiaogang

Guowei Ma  于2019年9月11日周三 下午7:07写道：

> Congratulations Zili !
>
> Best,
> Guowei
>
>
> Fabian Hueske  于2019年9月11日周三 下午7:02写道：
>
>> Congrats Zili Chen :-)
>>
>> Cheers, Fabian
>>
>> Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu > >:
>>
>>> Congrats Zili!
>>>
>>> Thanks,
>>> Biao /'bɪ.aʊ/
>>>
>>>
>>>
>>> On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:
>>>
 Congratulations!

 ---
 Oytun Tez

 *M O T A W O R D*
 The World's Fastest Human Translation Platform.
 oy...@motaword.com — www.motaword.com


 On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:

> Congratulations!
>
>
> Best,
>
> Jiayi Liao
>
>  Original Message
> *Sender:* Till Rohrmann
> *Recipient:* dev; user
> *Date:* Wednesday, Sep 11, 2019 17:22
> *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
>
> Hi everyone,
>
> I'm very happy to announce that Zili Chen (some of you might also
> know him as Tison Kun) accepted the offer of the Flink PMC to become a
> committer of the Flink project.
>
> Zili Chen has been an active community member for almost 16 months
> now. He helped pushing the Flip-6 effort over the finish line, ported a 
> lot
> of legacy code tests, removed a good part of the legacy code, contributed
> numerous fixes, is involved in the Flink's client API refactoring, drives
> the refactoring of Flink's HighAvailabilityServices and much more. Zili
> Chen also helped the community by PR reviews, reporting Flink issues,
> answering user mails and being very active on the dev mailing list.
>
> Congratulations Zili Chen!
>
> Best, Till
> (on behalf of the Flink PMC)
>

Re: error in Elasticsearch

[alaa]

Thank for your reply >> I run it >> and I think it run correctly now 



 



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

RE: [ANNOUNCE] Zili Chen becomes a Flink committer

[xingcanc]

Congratulations, Zili.

 

Best,

Xingcan

 

From: SHI Xiaogang  
Sent: Wednesday, September 11, 2019 7:43 AM
To: Guowei Ma 
Cc: Fabian Hueske ; Biao Liu ; Oytun Tez 
; bupt_ljy ; dev ; 
user ; Till Rohrmann 
Subject: Re: [ANNOUNCE] Zili Chen becomes a Flink committer

 

Congratulations!

 

Regards,

Xiaogang

 

Guowei Ma mailto:guowei@gmail.com> > 于2019年9月11日周三 
下午7:07写道：

Congratulations Zili !




Best,

Guowei

 

 

Fabian Hueske mailto:fhue...@gmail.com> > 于2019年9月11日周三 
下午7:02写道：

Congrats Zili Chen :-)

 

Cheers, Fabian

 

Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu mailto:mmyy1...@gmail.com> >:

Congrats Zili! 


 

Thanks,

Biao /'bɪ.aʊ/

 

 

 

On Wed, 11 Sep 2019 at 18:43, Oytun Tez mailto:oy...@motaword.com> > wrote:

Congratulations!


 

---

Oytun Tez

 

M O T A W O R D

The World's Fastest Human Translation Platform.

  oy...@motaword.com —   
www.motaword.com

 

 

On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy mailto:bupt_...@163.com> > wrote:

Congratulations!

 

Best,

Jiayi Liao

 

 Original Message  

Sender: Till Rohrmannmailto:trohrm...@apache.org> >

Recipient: devmailto:d...@flink.apache.org> >; 
usermailto:user@flink.apache.org> >

Date: Wednesday, Sep 11, 2019 17:22

Subject: [ANNOUNCE] Zili Chen becomes a Flink committer

 

Hi everyone,

 

I'm very happy to announce that Zili Chen (some of you might also know him as 
Tison Kun) accepted the offer of the Flink PMC to become a committer of the 
Flink project.

 

Zili Chen has been an active community member for almost 16 months now. He 
helped pushing the Flip-6 effort over the finish line, ported a lot of legacy 
code tests, removed a good part of the legacy code, contributed numerous fixes, 
is involved in the Flink's client API refactoring, drives the refactoring of 
Flink's HighAvailabilityServices and much more. Zili Chen also helped the 
community by PR reviews, reporting Flink issues, answering user mails and being 
very active on the dev mailing list.

 

Congratulations Zili Chen!

 

Best, Till 

(on behalf of the Flink PMC)

Job recovery from a checkpoint

[min.tan]

Hi,

We can get a job recovery from a save point nicely after a restart of our flink 
cluster using
bin/flink run -s :savepointPath [:runArgs]
The previous job states are recovered after this reload.
I expect I do something similar to recover a flink from a checkpoint location 
after a restart of our flink cluster (job manager and task manager) using
bin/flink run  –s  checkpointPath/_metadata  [:runArgs]
It seems that our reloaded job does not keep the previous states of the job.

Do I do something wrong? I suppose this is doable and no additional 
configuration is required?

Regards,

Min


E-mails can involve SUBSTANTIAL RISKS, e.g. lack of confidentiality, potential 
manipulation of contents and/or sender's address, incorrect recipient 
(misdirection), viruses etc. Based on previous e-mail correspondence with you 
and/or an agreement reached with you, UBS considers itself authorized to 
contact you via e-mail. UBS assumes no responsibility for any loss or damage 
resulting from the use of e-mails. 
The recipient is aware of and accepts the inherent risks of using e-mails, in 
particular the risk that the banking relationship and confidential information 
relating thereto are disclosed to third parties.
UBS reserves the right to retain and monitor all messages. Messages are 
protected and accessed only in legally justified cases.
For information on how UBS uses and discloses personal data, how long we retain 
it, how we keep it secure and your data protection rights, please see our 
Privacy Notice http://www.ubs.com/privacy-statement

Re: Job recovery from a checkpoint

[Yun Tang]

Hi Min

First of all, Flink could resume from an externalized checkpoint with same 
command as restoring from savepoint.

  *   Did you make the externalized checkpoint retained after job canceled?
  *   Did you really pass the correct checkpoint path (including chk-xxx 
folder) to the command line?

If you really pass the correct path, please check the jobmanager log to see 
what happened, did it restore from the checkpoint you want?

Best
Yun Tang

From: min@ubs.com 
Sent: Thursday, September 12, 2019 0:37
To: user@flink.apache.org 
Subject: Job recovery from a checkpoint


Hi,



We can get a job recovery from a save point nicely after a restart of our flink 
cluster using

bin/flink run -s :savepointPath [:runArgs]

The previous job states are recovered after this reload.

I expect I do something similar to recover a flink from a checkpoint location 
after a restart of our flink cluster (job manager and task manager) using

bin/flink run  –s  checkpointPath/_metadata  [:runArgs]

It seems that our reloaded job does not keep the previous states of the job.



Do I do something wrong? I suppose this is doable and no additional 
configuration is required?



Regards,



Min

Kafka Schema registry

[Lasse Nedergaard]

Hi. 
Do Flink have out of the Box Support for Kafka Schema registry for both sources 
and sinks?
If not, does anyone knows about a implementation we can build on so we can help 
make it general available in a future release. 

Med venlig hilsen / Best regards
Lasse Nedergaard

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Jark Wu]

Congratulations Zili!

Best,
Jark

On Wed, 11 Sep 2019 at 23:06,  wrote:

> Congratulations, Zili.
>
>
>
> Best,
>
> Xingcan
>
>
>
> *From:* SHI Xiaogang 
> *Sent:* Wednesday, September 11, 2019 7:43 AM
> *To:* Guowei Ma 
> *Cc:* Fabian Hueske ; Biao Liu ;
> Oytun Tez ; bupt_ljy ; dev <
> d...@flink.apache.org>; user ; Till Rohrmann <
> trohrm...@apache.org>
> *Subject:* Re: [ANNOUNCE] Zili Chen becomes a Flink committer
>
>
>
> Congratulations!
>
>
>
> Regards,
>
> Xiaogang
>
>
>
> Guowei Ma  于2019年9月11日周三 下午7:07写道：
>
> Congratulations Zili !
>
>
> Best,
>
> Guowei
>
>
>
>
>
> Fabian Hueske  于2019年9月11日周三 下午7:02写道：
>
> Congrats Zili Chen :-)
>
>
>
> Cheers, Fabian
>
>
>
> Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu :
>
> Congrats Zili!
>
>
>
> Thanks,
>
> Biao /'bɪ.aʊ/
>
>
>
>
>
>
>
> On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:
>
> Congratulations!
>
>
>
> ---
>
> Oytun Tez
>
>
>
> *M O T A W O R D*
>
> *The World's Fastest Human Translation Platform.*
>
> oy...@motaword.com — www.motaword.com
>
>
>
>
>
> On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:
>
> Congratulations!
>
>
>
> Best,
>
> Jiayi Liao
>
>
>
>  Original Message
>
> *Sender:* Till Rohrmann
>
> *Recipient:* dev; user
>
> *Date:* Wednesday, Sep 11, 2019 17:22
>
> *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
>
>
>
> Hi everyone,
>
>
>
> I'm very happy to announce that Zili Chen (some of you might also know
> him as Tison Kun) accepted the offer of the Flink PMC to become a committer
> of the Flink project.
>
>
>
> Zili Chen has been an active community member for almost 16 months now.
> He helped pushing the Flip-6 effort over the finish line, ported a lot of
> legacy code tests, removed a good part of the legacy code, contributed
> numerous fixes, is involved in the Flink's client API refactoring, drives
> the refactoring of Flink's HighAvailabilityServices and much more. Zili
> Chen also helped the community by PR reviews, reporting Flink issues,
> answering user mails and being very active on the dev mailing list.
>
>
>
> Congratulations Zili Chen!
>
>
>
> Best, Till
>
> (on behalf of the Flink PMC)
>
>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Hequn Cheng]

Congratulations!

Best, Hequn

On Thu, Sep 12, 2019 at 9:24 AM Jark Wu  wrote:

> Congratulations Zili!
>
> Best,
> Jark
>
> On Wed, 11 Sep 2019 at 23:06,  wrote:
>
> > Congratulations, Zili.
> >
> >
> >
> > Best,
> >
> > Xingcan
> >
> >
> >
> > *From:* SHI Xiaogang 
> > *Sent:* Wednesday, September 11, 2019 7:43 AM
> > *To:* Guowei Ma 
> > *Cc:* Fabian Hueske ; Biao Liu ;
> > Oytun Tez ; bupt_ljy ; dev <
> > d...@flink.apache.org>; user ; Till Rohrmann <
> > trohrm...@apache.org>
> > *Subject:* Re: [ANNOUNCE] Zili Chen becomes a Flink committer
> >
> >
> >
> > Congratulations!
> >
> >
> >
> > Regards,
> >
> > Xiaogang
> >
> >
> >
> > Guowei Ma  于2019年9月11日周三 下午7:07写道：
> >
> > Congratulations Zili !
> >
> >
> > Best,
> >
> > Guowei
> >
> >
> >
> >
> >
> > Fabian Hueske  于2019年9月11日周三 下午7:02写道：
> >
> > Congrats Zili Chen :-)
> >
> >
> >
> > Cheers, Fabian
> >
> >
> >
> > Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu  >:
> >
> > Congrats Zili!
> >
> >
> >
> > Thanks,
> >
> > Biao /'bɪ.aʊ/
> >
> >
> >
> >
> >
> >
> >
> > On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:
> >
> > Congratulations!
> >
> >
> >
> > ---
> >
> > Oytun Tez
> >
> >
> >
> > *M O T A W O R D*
> >
> > *The World's Fastest Human Translation Platform.*
> >
> > oy...@motaword.com — www.motaword.com
> >
> >
> >
> >
> >
> > On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:
> >
> > Congratulations!
> >
> >
> >
> > Best,
> >
> > Jiayi Liao
> >
> >
> >
> >  Original Message
> >
> > *Sender:* Till Rohrmann
> >
> > *Recipient:* dev; user
> >
> > *Date:* Wednesday, Sep 11, 2019 17:22
> >
> > *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
> >
> >
> >
> > Hi everyone,
> >
> >
> >
> > I'm very happy to announce that Zili Chen (some of you might also know
> > him as Tison Kun) accepted the offer of the Flink PMC to become a
> committer
> > of the Flink project.
> >
> >
> >
> > Zili Chen has been an active community member for almost 16 months now.
> > He helped pushing the Flip-6 effort over the finish line, ported a lot of
> > legacy code tests, removed a good part of the legacy code, contributed
> > numerous fixes, is involved in the Flink's client API refactoring, drives
> > the refactoring of Flink's HighAvailabilityServices and much more. Zili
> > Chen also helped the community by PR reviews, reporting Flink issues,
> > answering user mails and being very active on the dev mailing list.
> >
> >
> >
> > Congratulations Zili Chen!
> >
> >
> >
> > Best, Till
> >
> > (on behalf of the Flink PMC)
> >
> >
>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Rong Rong]

Congratulations Zili!

--
Rong

On Wed, Sep 11, 2019 at 6:26 PM Hequn Cheng  wrote:

> Congratulations!
>
> Best, Hequn
>
> On Thu, Sep 12, 2019 at 9:24 AM Jark Wu  wrote:
>
>> Congratulations Zili!
>>
>> Best,
>> Jark
>>
>> On Wed, 11 Sep 2019 at 23:06,  wrote:
>>
>> > Congratulations, Zili.
>> >
>> >
>> >
>> > Best,
>> >
>> > Xingcan
>> >
>> >
>> >
>> > *From:* SHI Xiaogang 
>> > *Sent:* Wednesday, September 11, 2019 7:43 AM
>> > *To:* Guowei Ma 
>> > *Cc:* Fabian Hueske ; Biao Liu ;
>> > Oytun Tez ; bupt_ljy ; dev <
>> > d...@flink.apache.org>; user ; Till Rohrmann <
>> > trohrm...@apache.org>
>> > *Subject:* Re: [ANNOUNCE] Zili Chen becomes a Flink committer
>> >
>> >
>> >
>> > Congratulations!
>> >
>> >
>> >
>> > Regards,
>> >
>> > Xiaogang
>> >
>> >
>> >
>> > Guowei Ma  于2019年9月11日周三 下午7:07写道：
>> >
>> > Congratulations Zili !
>> >
>> >
>> > Best,
>> >
>> > Guowei
>> >
>> >
>> >
>> >
>> >
>> > Fabian Hueske  于2019年9月11日周三 下午7:02写道：
>> >
>> > Congrats Zili Chen :-)
>> >
>> >
>> >
>> > Cheers, Fabian
>> >
>> >
>> >
>> > Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu <
>> mmyy1...@gmail.com>:
>> >
>> > Congrats Zili!
>> >
>> >
>> >
>> > Thanks,
>> >
>> > Biao /'bɪ.aʊ/
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:
>> >
>> > Congratulations!
>> >
>> >
>> >
>> > ---
>> >
>> > Oytun Tez
>> >
>> >
>> >
>> > *M O T A W O R D*
>> >
>> > *The World's Fastest Human Translation Platform.*
>> >
>> > oy...@motaword.com — www.motaword.com
>> >
>> >
>> >
>> >
>> >
>> > On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:
>> >
>> > Congratulations!
>> >
>> >
>> >
>> > Best,
>> >
>> > Jiayi Liao
>> >
>> >
>> >
>> >  Original Message
>> >
>> > *Sender:* Till Rohrmann
>> >
>> > *Recipient:* dev; user
>> >
>> > *Date:* Wednesday, Sep 11, 2019 17:22
>> >
>> > *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
>> >
>> >
>> >
>> > Hi everyone,
>> >
>> >
>> >
>> > I'm very happy to announce that Zili Chen (some of you might also know
>> > him as Tison Kun) accepted the offer of the Flink PMC to become a
>> committer
>> > of the Flink project.
>> >
>> >
>> >
>> > Zili Chen has been an active community member for almost 16 months now.
>> > He helped pushing the Flip-6 effort over the finish line, ported a lot
>> of
>> > legacy code tests, removed a good part of the legacy code, contributed
>> > numerous fixes, is involved in the Flink's client API refactoring,
>> drives
>> > the refactoring of Flink's HighAvailabilityServices and much more. Zili
>> > Chen also helped the community by PR reviews, reporting Flink issues,
>> > answering user mails and being very active on the dev mailing list.
>> >
>> >
>> >
>> > Congratulations Zili Chen!
>> >
>> >
>> >
>> > Best, Till
>> >
>> > (on behalf of the Flink PMC)
>> >
>> >
>>
>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Paul Lam]

Congratulations Zili！

Best,
Paul Lam

> 在 2019年9月12日，09:34，Rong Rong  写道：
> 
> Congratulations Zili!
> 
> --
> Rong
> 
> On Wed, Sep 11, 2019 at 6:26 PM Hequn Cheng  > wrote:
> Congratulations!
> 
> Best, Hequn
> 
> On Thu, Sep 12, 2019 at 9:24 AM Jark Wu  > wrote:
> Congratulations Zili!
> 
> Best,
> Jark
> 
> On Wed, 11 Sep 2019 at 23:06,  > wrote:
> 
> > Congratulations, Zili.
> >
> >
> >
> > Best,
> >
> > Xingcan
> >
> >
> >
> > *From:* SHI Xiaogang  > >
> > *Sent:* Wednesday, September 11, 2019 7:43 AM
> > *To:* Guowei Ma mailto:guowei@gmail.com>>
> > *Cc:* Fabian Hueske mailto:fhue...@gmail.com>>; Biao 
> > Liu mailto:mmyy1...@gmail.com>>;
> > Oytun Tez mailto:oy...@motaword.com>>; bupt_ljy 
> > mailto:bupt_...@163.com>>; dev <
> > d...@flink.apache.org >; user 
> > mailto:user@flink.apache.org>>; Till Rohrmann <
> > trohrm...@apache.org >
> > *Subject:* Re: [ANNOUNCE] Zili Chen becomes a Flink committer
> >
> >
> >
> > Congratulations!
> >
> >
> >
> > Regards,
> >
> > Xiaogang
> >
> >
> >
> > Guowei Ma mailto:guowei@gmail.com>> 
> > 于2019年9月11日周三 下午7:07写道：
> >
> > Congratulations Zili !
> >
> >
> > Best,
> >
> > Guowei
> >
> >
> >
> >
> >
> > Fabian Hueske mailto:fhue...@gmail.com>> 于2019年9月11日周三 
> > 下午7:02写道：
> >
> > Congrats Zili Chen :-)
> >
> >
> >
> > Cheers, Fabian
> >
> >
> >
> > Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu  > >:
> >
> > Congrats Zili!
> >
> >
> >
> > Thanks,
> >
> > Biao /'bɪ.aʊ/
> >
> >
> >
> >
> >
> >
> >
> > On Wed, 11 Sep 2019 at 18:43, Oytun Tez  > > wrote:
> >
> > Congratulations!
> >
> >
> >
> > ---
> >
> > Oytun Tez
> >
> >
> >
> > *M O T A W O R D*
> >
> > *The World's Fastest Human Translation Platform.*
> >
> > oy...@motaword.com  — www.motaword.com 
> > 
> >
> >
> >
> >
> >
> > On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  > > wrote:
> >
> > Congratulations!
> >
> >
> >
> > Best,
> >
> > Jiayi Liao
> >
> >
> >
> >  Original Message
> >
> > *Sender:* Till Rohrmannmailto:trohrm...@apache.org>>
> >
> > *Recipient:* devmailto:d...@flink.apache.org>>; 
> > usermailto:user@flink.apache.org>>
> >
> > *Date:* Wednesday, Sep 11, 2019 17:22
> >
> > *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
> >
> >
> >
> > Hi everyone,
> >
> >
> >
> > I'm very happy to announce that Zili Chen (some of you might also know
> > him as Tison Kun) accepted the offer of the Flink PMC to become a committer
> > of the Flink project.
> >
> >
> >
> > Zili Chen has been an active community member for almost 16 months now.
> > He helped pushing the Flip-6 effort over the finish line, ported a lot of
> > legacy code tests, removed a good part of the legacy code, contributed
> > numerous fixes, is involved in the Flink's client API refactoring, drives
> > the refactoring of Flink's HighAvailabilityServices and much more. Zili
> > Chen also helped the community by PR reviews, reporting Flink issues,
> > answering user mails and being very active on the dev mailing list.
> >
> >
> >
> > Congratulations Zili Chen!
> >
> >
> >
> > Best, Till
> >
> > (on behalf of the Flink PMC)
> >
> >

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[Becket Qin]

Congrats, Zili!

On Thu, Sep 12, 2019 at 9:39 AM Paul Lam  wrote:

> Congratulations Zili！
>
> Best,
> Paul Lam
>
> 在 2019年9月12日，09:34，Rong Rong  写道：
>
> Congratulations Zili!
>
> --
> Rong
>
> On Wed, Sep 11, 2019 at 6:26 PM Hequn Cheng  wrote:
>
>> Congratulations!
>>
>> Best, Hequn
>>
>> On Thu, Sep 12, 2019 at 9:24 AM Jark Wu  wrote:
>>
>>> Congratulations Zili!
>>>
>>> Best,
>>> Jark
>>>
>>> On Wed, 11 Sep 2019 at 23:06,  wrote:
>>>
>>> > Congratulations, Zili.
>>> >
>>> >
>>> >
>>> > Best,
>>> >
>>> > Xingcan
>>> >
>>> >
>>> >
>>> > *From:* SHI Xiaogang 
>>> > *Sent:* Wednesday, September 11, 2019 7:43 AM
>>> > *To:* Guowei Ma 
>>> > *Cc:* Fabian Hueske ; Biao Liu >> >;
>>> > Oytun Tez ; bupt_ljy ; dev <
>>> > d...@flink.apache.org>; user ; Till Rohrmann <
>>> > trohrm...@apache.org>
>>> > *Subject:* Re: [ANNOUNCE] Zili Chen becomes a Flink committer
>>> >
>>> >
>>> >
>>> > Congratulations!
>>> >
>>> >
>>> >
>>> > Regards,
>>> >
>>> > Xiaogang
>>> >
>>> >
>>> >
>>> > Guowei Ma  于2019年9月11日周三 下午7:07写道：
>>> >
>>> > Congratulations Zili !
>>> >
>>> >
>>> > Best,
>>> >
>>> > Guowei
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > Fabian Hueske  于2019年9月11日周三 下午7:02写道：
>>> >
>>> > Congrats Zili Chen :-)
>>> >
>>> >
>>> >
>>> > Cheers, Fabian
>>> >
>>> >
>>> >
>>> > Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu <
>>> mmyy1...@gmail.com>:
>>> >
>>> > Congrats Zili!
>>> >
>>> >
>>> >
>>> > Thanks,
>>> >
>>> > Biao /'bɪ.aʊ/
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:
>>> >
>>> > Congratulations!
>>> >
>>> >
>>> >
>>> > ---
>>> >
>>> > Oytun Tez
>>> >
>>> >
>>> >
>>> > *M O T A W O R D*
>>> >
>>> > *The World's Fastest Human Translation Platform.*
>>> >
>>> > oy...@motaword.com — www.motaword.com
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:
>>> >
>>> > Congratulations!
>>> >
>>> >
>>> >
>>> > Best,
>>> >
>>> > Jiayi Liao
>>> >
>>> >
>>> >
>>> >  Original Message
>>> >
>>> > *Sender:* Till Rohrmann
>>> >
>>> > *Recipient:* dev; user
>>> >
>>> > *Date:* Wednesday, Sep 11, 2019 17:22
>>> >
>>> > *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
>>> >
>>> >
>>> >
>>> > Hi everyone,
>>> >
>>> >
>>> >
>>> > I'm very happy to announce that Zili Chen (some of you might also know
>>> > him as Tison Kun) accepted the offer of the Flink PMC to become a
>>> committer
>>> > of the Flink project.
>>> >
>>> >
>>> >
>>> > Zili Chen has been an active community member for almost 16 months now.
>>> > He helped pushing the Flip-6 effort over the finish line, ported a lot
>>> of
>>> > legacy code tests, removed a good part of the legacy code, contributed
>>> > numerous fixes, is involved in the Flink's client API refactoring,
>>> drives
>>> > the refactoring of Flink's HighAvailabilityServices and much more. Zili
>>> > Chen also helped the community by PR reviews, reporting Flink issues,
>>> > answering user mails and being very active on the dev mailing list.
>>> >
>>> >
>>> >
>>> > Congratulations Zili Chen!
>>> >
>>> >
>>> >
>>> > Best, Till
>>> >
>>> > (on behalf of the Flink PMC)
>>> >
>>> >
>>>
>>
>

Re: [ANNOUNCE] Zili Chen becomes a Flink committer

[zhijiang]

Congratulations Zili！
--
From:Becket Qin 
Send Time:2019年9月12日(星期四) 03:43
To:Paul Lam 
Cc:Rong Rong ; dev ; user 

Subject:Re: [ANNOUNCE] Zili Chen becomes a Flink committer

Congrats, Zili!
On Thu, Sep 12, 2019 at 9:39 AM Paul Lam  wrote:
Congratulations Zili！

Best,
Paul Lam

在 2019年9月12日，09:34，Rong Rong  写道：
Congratulations Zili!

--
Rong
On Wed, Sep 11, 2019 at 6:26 PM Hequn Cheng  wrote:
Congratulations!

Best, Hequn
On Thu, Sep 12, 2019 at 9:24 AM Jark Wu  wrote:
Congratulations Zili!

 Best,
 Jark

 On Wed, 11 Sep 2019 at 23:06,  wrote:

 > Congratulations, Zili.
 >
 >
 >
 > Best,
 >
 > Xingcan
 >
 >
 >
 > *From:* SHI Xiaogang 
 > *Sent:* Wednesday, September 11, 2019 7:43 AM
 > *To:* Guowei Ma 
 > *Cc:* Fabian Hueske ; Biao Liu ;
 > Oytun Tez ; bupt_ljy ; dev <
 > d...@flink.apache.org>; user ; Till Rohrmann <
 > trohrm...@apache.org>
 > *Subject:* Re: [ANNOUNCE] Zili Chen becomes a Flink committer
 >
 >
 >
 > Congratulations!
 >
 >
 >
 > Regards,
 >
 > Xiaogang
 >
 >
 >
 > Guowei Ma  于2019年9月11日周三 下午7:07写道：
 >
 > Congratulations Zili !
 >
 >
 > Best,
 >
 > Guowei
 >
 >
 >
 >
 >
 > Fabian Hueske  于2019年9月11日周三 下午7:02写道：
 >
 > Congrats Zili Chen :-)
 >
 >
 >
 > Cheers, Fabian
 >
 >
 >
 > Am Mi., 11. Sept. 2019 um 12:48 Uhr schrieb Biao Liu :
 >
 > Congrats Zili!
 >
 >
 >
 > Thanks,
 >
 > Biao /'bɪ.aʊ/
 >
 >
 >
 >
 >
 >
 >
 > On Wed, 11 Sep 2019 at 18:43, Oytun Tez  wrote:
 >
 > Congratulations!
 >
 >
 >
 > ---
 >
 > Oytun Tez
 >
 >
 >
 > *M O T A W O R D*
 >
 > *The World's Fastest Human Translation Platform.*
 >
 > oy...@motaword.com — www.motaword.com
 >
 >
 >
 >
 >
 > On Wed, Sep 11, 2019 at 6:36 AM bupt_ljy  wrote:
 >
 > Congratulations!
 >
 >
 >
 > Best,
 >
 > Jiayi Liao
 >
 >
 >
 >  Original Message
 >
 > *Sender:* Till Rohrmann
 >
 > *Recipient:* dev; user
 >
 > *Date:* Wednesday, Sep 11, 2019 17:22
 >
 > *Subject:* [ANNOUNCE] Zili Chen becomes a Flink committer
 >
 >
 >
 > Hi everyone,
 >
 >
 >
 > I'm very happy to announce that Zili Chen (some of you might also know
 > him as Tison Kun) accepted the offer of the Flink PMC to become a committer
 > of the Flink project.
 >
 >
 >
 > Zili Chen has been an active community member for almost 16 months now.
 > He helped pushing the Flip-6 effort over the finish line, ported a lot of
 > legacy code tests, removed a good part of the legacy code, contributed
 > numerous fixes, is involved in the Flink's client API refactoring, drives
 > the refactoring of Flink's HighAvailabilityServices and much more. Zili
 > Chen also helped the community by PR reviews, reporting Flink issues,
 > answering user mails and being very active on the dev mailing list.
 >
 >
 >
 > Congratulations Zili Chen!
 >
 >
 >
 > Best, Till
 >
 > (on behalf of the Flink PMC)
 >
 >

Re: Checkpointing is not performing well

[Vijay Bhaskar]

I meant upper limit w.r.t resources you are using. Even if you
increase resources, Spiking data is always a problem which anyways you need
to take care of. Best thing is to add more back pressure from source.

Regards
Bhaskar

On Wed, Sep 11, 2019 at 1:43 PM Fabian Hueske  wrote:

> Hi,
>
> There is no upper limit for state size in Flink. There are applications
> with 10+ TB state.
> However, it is natural that checkpointing time increases with state size
> as more data needs to be serialized (in case of FSStateBackend) and written
> to stable storage.
> (The same is btw true for recovery when the state needs to be loaded back.)
>
> There are a few tricks to reduce checkpointing time like using incremental
> checkpoints which you tried already.
> You can also scale out the application to use more machines and therefore
> bandwidth + CPU (for serialization) during checkpoints.
>
> Fabian
>
> Am Mi., 11. Sept. 2019 um 09:38 Uhr schrieb Ravi Bhushan Ratnakar <
> ravibhushanratna...@gmail.com>:
>
>> What is the upper limit of checkpoint size of Flink System?
>>
>> Regards,
>> Ravi
>>
>> On Wed 11 Sep, 2019, 06:48 Vijay Bhaskar, 
>> wrote:
>>
>>> You crossed  the upper limits of the check point system of Flink a way
>>> high. Try to distribute events equally over time by adding some sort of
>>> controlled back pressure after receiving data from kinesis streams.
>>> Otherwise the spike coming during 5 seconds time would always create
>>> problems. Tomorrow it may double so best solution in your case is to
>>> deliver at configurable constant rate after receiving messages from kinesis
>>> streams. Otherwise i am sure its always the problem whatever the kind of
>>> streaming engine you use. Tune your configuration to get the optimal rate
>>> so that flink checkpoint state is healthier.
>>>
>>> Regards
>>> Bhaskar
>>>
>>> On Tue, Sep 10, 2019 at 11:16 PM Ravi Bhushan Ratnakar <
>>> ravibhushanratna...@gmail.com> wrote:
>>>
 @Rohan - I am streaming data to kafka sink after applying business
 logic. For checkpoint, I am using s3 as a distributed file system. For
 local recovery, I am using Optimized iops ebs volume.

 @Vijay - I forget to mention that incoming data volume is ~ 10 to 21GB
 per minute compressed(lz4) avro message. Generally 90% correlated events
 come within 5 seconds and 10% of the correlated events get extended to 65
 minute. Due to this business requirement, the state size keep growing till
 65 minutes, after that the state size becomes more or less stable. As the
 state size is growing and is around 350gb at peak load, checkpoint is not
 able to complete within 1 minutes. I want to check as quick as possible
 like every 5 second.

 Thanks,
 Ravi


 On Tue 10 Sep, 2019, 11:37 Vijay Bhaskar, 
 wrote:

> For me task count seems to be huge in number with the mentioned
> resource count. To rule out the possibility of issue with state backend 
> can
> you start writing sink data as  , i.e., data ignore sink. 
> And
> try whether you could run it for longer duration without any issue. You 
> can
> start decreasing the task manager count until you find descent count of it
> without having any side effects. Use that value as task manager count and
> then start adding your state backend. First you can try with Rocks DB. 
> With
> reduced task manager count you might get good results.
>
> Regards
> Bhaskar
>
> On Sun, Sep 8, 2019 at 10:15 AM Rohan Thimmappa <
> rohan.thimma...@gmail.com> wrote:
>
>> Ravi, have you looked at the io operation(iops) rate of the disk? You
>> can monitoring the iops performance and tune it accordingly with your 
>> work
>> load. This helped us in our project when we hit the wall tuning prototype
>> much all the parameters.
>>
>> Rohan
>>
>>
>> --
>> *From:* Ravi Bhushan Ratnakar 
>> *Sent:* Saturday, September 7, 2019 5:38 PM
>> *To:* Rafi Aroch
>> *Cc:* user
>> *Subject:* Re: Checkpointing is not performing well
>>
>> Hi Rafi,
>>
>> Thank you for your quick response.
>>
>> I have tested with rocksdb state backend. Rocksdb required
>> significantly more taskmanager to perform as compare to filesystem state
>> backend. The problem here is that checkpoint process is not fast enough 
>> to
>> complete.
>>
>> Our requirement is to do checkout as soon as possible like in 5
>> seconds to flush the output to output sink. As the incoming data rate is
>> high, it is not able to complete quickly. If I increase the checkpoint
>> duration, the state size grows much faster and hence takes much longer 
>> time
>> to complete checkpointing. I also tried to use AT LEAST ONCE mode, but 
>> does
>> not improve much. Adding more taskmanager to increase parallelism also 
>> does
>>>