Our setup involves AWS IAM roles when with permanent access_key and
access_secret we need to assume specific role (ie getting temporary credentials
to use AWS resources).
I was wondering what would be the best way handling this, ie how to set
fs.s3n.awsAccessKeyId and fs.s3n.awsSecretAccessKey p
Gave it a try, but does not seem to help. Is it working for you?
Thanks
Sent from my iPhone
> On Dec 3, 2015, at 6:11 PM, Vladimir Stoyak wrote:
>
> As far as I know "auto.offset.reset" what to do if offset it not available or
> out of bound?
>
> Vladimir
>
e
offset is only unique per partition. You could modify the offsets in
the Zookeeper state but you really have to know what you're doing
then.
Best regards,
Max
On Thu, Dec 3, 2015 at 4:01 PM, Vladimir Stoyak wrote:
> I see that Flink 0.10.1 now supports Keyed Schemas which allows us
I see that Flink 0.10.1 now supports Keyed Schemas which allows us to rely on
Kafka topics set to "compact" retention for data persistence.
In our topology we wanted to set some topics with Log Compactions enabled and
read topic from the beginning when the topology starts or component recovers.
We, were also trying to address session windowing but took slightly different
approach as to what window we place the event into.
We did not want "triggering event" to be purged as part of the window it
triggered, but instead to create a new window for it and have the old window to
fire and pu
I know I can use broadcast, but was wondering if there is a better way
DataStream control_stream = env.addSource(new
FlinkKafkaConsumer082(control_topic, new
AvroDeserializationSchema(Model.class), properties)).broadcast();
On Tuesday, November 17, 2015 2:45 PM, Vladimir Stoyak
wrote
Not that I necessarily need that for this particular example, but is there a
Global State available?
IE, how can I make a state available across all parallel instances of an
operator?
On Tuesday, November 17, 2015 1:49 PM, Vladimir Stoyak
wrote:
Perfect! It does explain my
17, 2015 at 1:11 PM, Vladimir Stoyak wrote:
My model DataStream is not keyed and does not have any windows, only the main
stream has windows and apply function
I have two Kafka Streams, one for events and one for model
DataStream model_stream = env.addSource(new
FlinkKafkaConsumer082(model_topic
uts to make sure that the records are
properly routed...
Greetings,Stephan
On Tue, Nov 17, 2015 at 12:29 PM, Vladimir Stoyak wrote:
Got stuck a bit with CoFlatMapFunction. It seems to work fine if I place it on
the DataStream before window but fails if placed after window's “apply”
funct
Got stuck a bit with CoFlatMapFunction. It seems to work fine if I place it on
the DataStream before window but fails if placed after window's “apply”
function.I was testing two streams, main “Features” on flatMap1 constantly
ingesting data and control stream “Model” on flatMap2 changing the mod
10 matches
Mail list logo
