There are a lot of variables. How many cores are allocated, how much ram, etc.
there are companies doing billions of events per day and more.
Tell your boss it has proven to have extremely flat horizontal scaling. Meaning
you can get it to process almost any number given sufficient hardware.
You may want to union the 3 streams prior to the process function if they are
independently processed.
Michael
> On Feb 22, 2019, at 9:15 AM, Oytun Tez wrote:
>
> Hi everyone!
>
> I've been struggling with an implementation problem in the last days, which I
> am almost sure caused by my mi
gt;> Internal suggests private API to me, so I am asking if the docs are out of
>> date, or I am missing some library or such. I am using Java 8.0 not Scala.
>>
>> Michael Latta
>>
I would really like to see the job manager show metrics on state size, not just
io per task. Is there a way to do that now, or is the metric there, and just
needs some UI Ewok to show it?
Michael
Sent from my iPad
If you restart the job each month you can build the string dynamically. If you
want data to flow into the log based on a date in the record you will need to
do something fancier. I have not used the casandra connector so I can’t help on
the details. Can you subclass the connector and build the q
Not knowing the library or the config needs I do not have a suggestion. If the
config is accumulated from inputs and needs to see all inputs I would suggest
setting parallelism to 1 as an experiment, but it would need a redesign to run
in parallel.
Michael
Sent from my iPad
> On Apr 26, 2018
Only the anonymous FlatMapFunction instance is sent to the TaskManager. Move
the static field to that class.
Michael
Sent from my iPad
> On Apr 25, 2018, at 10:42 PM, Soheil Pourbafrani
> wrote:
>
> I run a code using Flink Java API that gets some bytes from Kafka and parses
> it following
Thanks for the suggestion. The task manager is configured for 8GB of heap, and
gets to about 8.3 total. Other java processes (job manager and Kafka). Add a
few more. I will check it again but the instances have 16GB same as my laptop
that completes the test in <90 min.
Michael
Sent from my iP
I would like to “join” several streams (>3) in a custom operator. Is this
feasible in Flink?
Michael
B, the state will be for key B.
>
> Best,
> Fabian
>
> [1]
> https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/state/state.html#keyed-state
>
> 2018-04-05 14:08 GMT+02:00 Michael Latta :
>> Thanks for the clarification. I was just trying to understand
Thanks for the clarification. I was just trying to understand the intended
behavior. It would have been nice if Flink tracked state for downstream
operators by key, but I can do that with a map in the downstream functions.
Michael
Sent from my iPad
> On Apr 5, 2018, at 2:30 AM, Fabian Hueske
11 matches
Mail list logo
