Hi Xingcan,
that's my bad, I was thinking of scatter-gather iterations in my previous
reply. You're right, in VertexCentricIteration a vertex is only active in
the next superstep if it has received at least one message in the current
superstep. Updating its value does not impact the activation. Th
I want to know the behavior of flink streaming systems during daylight
saving changes in multiple timezones. As streaming systems may in such
timezones.
Is there any built-in support is needed ? Can anybody answer ?
Thanks in advance
--Swapnil
Hi Dmitry,
Technically, from the looks of the internal code around
`OperatorStateRepartitioner`, I think it is certainly possible to be pluggable.
Right now it is just hard coded to use a round-robin repartitioner
implementation as default.
However, I’m not sure of the plans in exposing this to
Hi Greg,
I also found that in VertexCentricIteration.java, the message set is taken
as the workset while the vertex set is taken as the delta for solution set.
By doing like that, the setNewVertex method will not actually active a
vertex. In other words, if no message is generated (the workset is
I am assuming it's a bad idea to have DB operations in a map? May be a
better way is to chain the pipelines. Is that possible? eg:
workflow A ends -> workflow B starts
On Mon, Feb 13, 2017 at 12:38 PM, Abhishek Singh <
abhis...@tetrationanalytics.com> wrote:
> You can keep adding stages, but the
You can keep adding stages, but then your sink is no more a sink - it would
have transformed into a map or a flatmap !
On Mon, Feb 13, 2017 at 12:34 PM Mohit Anchlia
wrote:
> Is it possible to further add aggregation after the sink task executes? Or
> is the sink the last stage of the workflow?
Is it possible to further add aggregation after the sink task executes? Or
is the sink the last stage of the workflow? Is this flow possible?
start stream -> transform -> load (sink) -> mark final state as loaded in a
table after all the load was successful in previous state (sink)
Hi,
It looks impossible to implement a keyed state with operator state now.
I know it sounds like "just use a keyed state", but latter requires
updating it on every value change as opposed to operator state and thus can
be expensive (especially if you have to deal with mutable structures inside
w
Yes, but that information would have to "bubble" up from the downstream
operator to the source, which is not possible right now.
On Sun, 12 Feb 2017 at 17:15 Jonas wrote:
> For 2: You can also NOT read the Source (i.e. Kafka) while doing that. This
> way you don't have to buffer.
>
>
>
> --
> Vi
Just to clarify, is Flink designed to allow submitting multiple jobs from a
single program class when using a YARN cluster? I wasn't sure based on the
documentation.
Cheers,
Geoffrey
On Thu, Feb 9, 2017 at 6:34 PM Geoffrey Mon wrote:
> Hello all,
>
> I'm running a Flink plan made up of multiple
Hi,
My case is very similar to what is described in this link of Spark:
http://spark.apache.org/docs/latest/sql-programming-guide.html#programmatically-specifying-the-schema
I hope this clarifies it.
Thanks,
Luqman
On Mon, Feb 13, 2017 at 12:04 PM, Tzu-Li (Gordon) Tai
wrote:
> Hi Luqman,
>
>
Hi Greg,
Thanks for your attention.
It takes me a little time to read the old PR on FLINK-1885. Though
the VertexCentricIteration, as well as its related classes, has been
refactored, I understand what Markus want to achieve.
I am not sure if using a bulk iteration instead of a delta one could
e
Thanks a lot! I feel very happy and will try help the Flink community as good
as I can :-)
Best,
Stefan
> Am 10.02.2017 um 11:00 schrieb Ufuk Celebi :
>
> Hey everyone,
>
> I'm very happy to announce that the Flink PMC has accepted Stefan
> Richter to become a committer of the Apache Flink pr
13 matches
Mail list logo
