Question about Watermarks within a KeyedProcessFunction

2020-06-26 Thread Marco Villalobos
My source is a Kafka topic. I am using Event Time. I assign the event time with an AscendingTimestampExtractor I noticed when debugging that in the KeyedProcessFunction that after my highest known event time of: 2020-06-23T00:46:30.000Z the processElement method had a watermark with an impossibl

Re: datadog failed to send report

2020-06-26 Thread seeksst
原始邮件 发件人:seeksstseek...@163.com 收件人:Fanbin bufanbin...@coinbase.com 发送时间:2020年6月26日(周五) 23:36 主题:Re: datadog failed to send report Hi, I’m sorry for not explaining it clearly and misread the exception. log4j.logger.org.apache.flink.metrics.datadog.DatadogHttpClient=ERROR log4j.logger.org.apache.f

Re: Performance issue associated with managed RocksDB memory

2020-06-26 Thread Yu Li
To clarify, that my questions were all against the very original issue instead of the WordCount job. The timers come from the window operator you mentioned as the source of the original issue: === bq. If I create a Flink job that has a single "heavy" operator

Re: Performance issue associated with managed RocksDB memory

2020-06-26 Thread Andrey Zagrebin
Hi Juha, > I can also submit the more complex test with the bigger operator and and a > window operator. There's just gonna be more code to read. Can I attach a > file here or how should I submit a larger chuck of code? You can just attach the file with the code. > 2. I'm not sure what would / s

Absolute latency with Latency Tracking

2020-06-26 Thread Arti Pande
Hi Given an SLA of some x milliseconds of end-to-end latency, is there a way to know the end-to-end latency at record level (time taken by a record to be processed by the entire pipeline and written to sink) for a flink job? This