Hi Jingsong,
Thanks for the reply! The following is our code snippet for creating the
log stream. Our messages are in thrift format. We use a customized
serializer for serializing/deserializing messages ( see
https://github.com/apache/flink/pull/8067 for the implementation) . Given
that, how shal
+flink-user
On Wed, Jun 5, 2019 at 9:58 AM Yu Yang wrote:
> Thanks for the reply! In flink-table-planner, TimeIndicatorTypeInfo is an
> internal class that cannot be referenced from application. I got "cannot
> find symbol" error when I tried to use it. I have also tried to use "
> SqlTimeTypeI
Hi @Yu Yang:
Time-based operations such as windows in both the Table API and SQL require
information about the notion of time and its origin. Therefore, tables can
offer
logical time attributes for indicating time and accessing corresponding
timestamps
in table programs.[1]
This mean Window ca
Elias Thanks for your reply. In this case,
*When # of Kafka consumers = # of partitions, and I use setParallelism(>1),
something like this
'messageSteam.rebalance().map(lamba).setParallelism(3).print()'
*
If checkpointing is enabled, I assume Flink will commit the offsets in the
'right order' d
There is no such concept in Flink. Flink tracks offsets in its
checkpoints. It can optionally commit offsets to Kafka, but that is only
for reporting purposes. If you wish to lower the number of records that
get reprocessed in the case of a restart, then you must lower the
checkpoint interval.
Hi mates, I’m trying to configure my job to retain checkpoints on it’s
cancellation and got some troubles.
I got the following args why not to use savepoints:
1. we already have all the job state on DFS in checkpoints directory
2. I can multiply size of the state on 2, when stopping the job, b
Ok, I will do that.
On Wed, Jun 5, 2019, 8:25 AM Chesnay Schepler wrote:
> Can you provide us the jobmanager logs?
>
> After the first restart the JM should have started deleting older
> checkpoints as new ones were created.
> After the second restart the JM should have recovered all 10 checkpoi
Can you provide us the jobmanager logs?
After the first restart the JM should have started deleting older
checkpoints as new ones were created.
After the second restart the JM should have recovered all 10
checkpoints, start from the latest, and start pruning old ones as new
ones were created.
Any one?
On Tue, Jun 4, 2019, 2:41 PM Vishal Santoshi
wrote:
> The above is flink 1.8
>
> On Tue, Jun 4, 2019 at 12:32 PM Vishal Santoshi
> wrote:
>
>> I had a sequence of events that created this issue.
>>
>> * I started a job and I had the state.checkpoints.num-retained: 5
>>
>> * As expected
