The design looks great - it solves for very diverse deployment modes, allows
for heterogeneous TMs, and promotes job isolation.
Some feedback:
*Dispatcher*
The dispatcher concept here expands nicely on what was introduced in the Mesos
design doc (MESOS-1984). The most significant difference be
@Gyula: This is documented in the JavaDoc of the `allowedLateness(..)`
method and in the docs:
https://ci.apache.org/projects/flink/flink-docs-master/apis/streaming/windows.html#dealing-with-late-data
@Kevin: Thanks for the explanation, I'll get back to you soon (sort of
in a rush).
Cheers,
Max
When running "mvn clean verify" with Hadoop version 2.6.1 the
Zookeeper/Leader Election tests fail with this:
java.lang.NoSuchMethodError:
org.apache.curator.utils.PathUtils.validatePath(Ljava/lang/String;)Ljava/lang/String;
at
org.apache.curator.framework.imps.NamespaceImpl.(NamespaceImpl.java:37
Thanks!
On Fri, Jul 29, 2016 at 11:43 AM, Gordon Tai (戴資力) wrote:
> Hi Max,
>
> Sure, I was planning to do so, but wanted to see if it was a reasonable
> feature to add before opening a JIRA :)
> Here's the new JIRA: https://issues.apache.org/jira/browse/FLINK-4280
>
> Regards,
> Gordon
>
> On Fr
Chesnay Schepler created FLINK-4284:
---
Summary: DataSet/CEP link to non-existant "Linking with Flink"
section
Key: FLINK-4284
URL: https://issues.apache.org/jira/browse/FLINK-4284
Project: Flink
Chesnay Schepler created FLINK-4283:
---
Summary: ExecutionGraphRestartTest fails
Key: FLINK-4283
URL: https://issues.apache.org/jira/browse/FLINK-4283
Project: Flink
Issue Type: Bug
Affec
Aljoscha Krettek created FLINK-4282:
---
Summary: Add Offset Parameter to WindowAssigners
Key: FLINK-4282
URL: https://issues.apache.org/jira/browse/FLINK-4282
Project: Flink
Issue Type: Impro
About processing time and timestamps:
The timestamp is either set in the source of in an
in-between TimestampAssigner that can be used with
DataStream.assignTimestampsAndWatermarks(). However, the timestamp in the
element is normally not a "processing-time timestamp". I think it might
make sense t
Timo Walther created FLINK-4281:
---
Summary: Wrap all Calcite Exceptions in Flink Exceptions
Key: FLINK-4281
URL: https://issues.apache.org/jira/browse/FLINK-4281
Project: Flink
Issue Type: Bug
Hi Max,
Sure, I was planning to do so, but wanted to see if it was a reasonable
feature to add before opening a JIRA :)
Here's the new JIRA: https://issues.apache.org/jira/browse/FLINK-4280
Regards,
Gordon
On Fri, Jul 29, 2016 at 4:03 PM, Maximilian Michels wrote:
> Hi Tai,
>
> Should definite
Tzu-Li (Gordon) Tai created FLINK-4280:
--
Summary: New Flink-specific option to set starting position of
Kafka consumer without respecting external offsets in ZK / Broker
Key: FLINK-4280
URL: https://issues.ap
Chesnay Schepler created FLINK-4279:
---
Summary: [py] Set flink dependencies to provided
Key: FLINK-4279
URL: https://issues.apache.org/jira/browse/FLINK-4279
Project: Flink
Issue Type: Impro
Goodmorning :-),
Thank you for your answer. Let me explain my problem more thoroughly
(maybe other options are possible here, not necessary with allowedLateness).
The most compact description of my problem would be Stream Enrichment.
More concrete, suppose I have two streams, where I want to
Hi Max,
So if I understand correctly the window operators now, by default, discard
late elements?
Is this documented somewhere?
Gyula
Maximilian Michels ezt írta (időpont: 2016. júl. 29., P,
10:40):
> Hi!
>
> I'm not sure whether I understand your question. The purpose of Event
> Time is to b
Hi!
I'm not sure whether I understand your question. The purpose of Event
Time is to be able to process out-of-order events. Do you want to
discard late elements? In the upcoming Flink 1.1.0 you can set the
`allowedLateness` on a windowed stream. The default is 0, so late
elements are discarded; l
Hi Tai,
Should definitely be possible. Would you mind opening a JIRA issue
with the description you posted?
Thanks,
Max
On Thu, Jul 28, 2016 at 11:16 AM, Tai Gordon wrote:
> Hi Kevin,
>
> Just a re-clarification: for Kafka 0.9 it would be “earliest”, & “smallest”
> for the older Kafka 0.8.
>
>
16 matches
Mail list logo
