Hi Guys,
Has anyone used flink-connector-cassadra with cassandra version >= 3.X? I
see that the cassandra version support is only 2.2.5 in the latest branch.
https://github.com/apache/flink/blob/master/flink-connectors/flink-connector-cassandra/pom.xml#L40
is there a way to pass these properties t
Hi there,
I have a flink stream from kafka writing to Cassandra. We use monthly
tables in Cassandra to avoid TTL and tombstones that come with it. Tables
would be like table_05_2018, table_06_2018 and so on. How do I dynamically
register this table name in the following snippet?
CassandraSink
.ad
Hi All,
Sorry for the miscommunication. I'm not using 0.8. I'm using latest
available flink-kafka client. I don't see my app registered as a consumer
group. I wanted to know if there is a way to monitor Kafka offsets.
Thanks,
Sandeep
On Apr 23, 2017 9:38 AM, "Stephan Ewen" wrote:
> Since it is
On Mon, Mar 13, 2017 at 11:22 AM, Meghashyam Sandeep V <
vr1meghash...@gmail.com> wrote:
> Thanks Suneel. Exactly what I was looking for.
>
> On Mon, Mar 13, 2017 at 10:31 AM, Suneel Marthi
> wrote:
>
>> For an example implementation using Flink, check out
>>
nt in time.
>>
>> On Mon, Mar 13, 2017 at 1:25 PM, Meghashyam Sandeep V <
>> vr1meghash...@gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I'm trying to use Flink for a use case where I would want to see my top
>>> selling products in t
Hi All,
I'm trying to use Flink for a use case where I would want to see my top
selling products in time windows in near real time (windows of size 1-2
mins if fine). I guess this is the most common use case to use streaming
apis in e-commerce. I see that I can iterate over records in a windowed
s
I spoke to one of the representatives in AWS EMR team last week. They
mentioned that they usually practice a 4 week cool down period. Hopefully
we will get Flink 1.2 in the next week.
Thanks,
Sandeep
On Mar 6, 2017 9:27 AM, "Chen Qin" wrote:
> EMR is a team within Amazon Web Services, to them,
Hi there,
Has anyone tried using flink interpreter in Zeppelin using AWS EMR? I
tried creating a new interpreter using host as 'localhots' and port '6123'
which didn't seem to work.
Thanks,
Sandeep
Hi Guys,
I currently run flink 1.1.4 streaming jobs in EMR in AWS with yarn. I
understand that EMR supports auto scaling but Flink doesn't. Is there a
plan for this support in 1.2.
Thanks,
Sandeep
amps, in order to measure the latency
> of specific records.
>
> Stephan
>
>
> On Fri, Dec 16, 2016 at 5:02 PM, Meghashyam Sandeep V <
> vr1meghash...@gmail.com> wrote:
>
>> Hi Stephan,
>>
>> Thanks for your answer. Is there a way to get the metrics such a
t; operator, i.e., each operator holds a full copy of the map.
> On the other hand, you do not need to shuffle the data because each
> parallel task can do the look-up.
> If your map is small, this would be the preferred approach.
>
> Best, Fabian
>
> 2016-12-21 18:46 GMT+01:00 Meghash
ink takes care of
> checkpointing the state and restoring it in case of a failure.
> In fact, the operator state is persisted in the state backends you
> mentioned before.
>
> Best, Fabian
>
> 2016-12-21 15:02 GMT+01:00 Meghashyam Sandeep V :
>
>> Hi Fabian,
>>
>>
> I'm sorry but I think I do not understand your question.
> What do you mean by static or dynamic look ups? Do you want to access an
> external data store and cache data?
>
> Can you give a bit more detail about your use?
>
> Best, Fabian
>
> 2016-12-20 23:0
Hi there,
I know that there are various state backends to persist state. Is there a
similar way to persist static/dynamic look ups and use them while streaming
the data in Flink?
Thanks,
Sandeep
Runtime error is because you have non-serializable code in your 'map'
operator.
DataStream stockPrices = streamExecEnv.addSource(new
LookupStockPrice(stockSymbol));
stockPrices.print();
The approach that you took will create infinite stockprice sources inside
the ma
ally be helpful to collect some jobs
> in a form of "evaluation suite".
>
> Stephan
>
>
>
> On Thu, Dec 15, 2016 at 6:11 PM, Meghashyam Sandeep V <
> vr1meghash...@gmail.com> wrote:
>
>> Hi There,
>>
>> We are evaluating Flink strea
Hi There,
We are evaluating Flink streaming for real time data analysis. I have my
flink job running in EMR with Yarn. What are the possible benchmarking
tools that work best with Flink? I couldn't find this information in the
Apache website.
Thanks,
Sandeep
1. If we have multiple sources, can the streams be parallelized ?
2. Can we have multiple sinks as well?
On Dec 14, 2016 10:46 PM, wrote:
> Got it. Thanks!
>
> On Dec 15, 2016, at 02:58, Jamie Grier wrote:
>
> Ahh, sorry, for #2: A single Flink job can have as many sources as you
> like. They c
Hi There,
I have a streaming job which has source as Kafka and sink as Cassandra. I
have a use case where I wouldn't want to write some events to Cassandra
when there are more than 100 events for a given 'id' (field in my Pojo) in
5mins. Is this a good usecase for SlidingWindows? Can I get the sli
eature to Flink's Cassandra Sink.
>
> Cheers,
> Till
>
> On Mon, Dec 12, 2016 at 4:30 PM, Meghashyam Sandeep V <
> vr1meghash...@gmail.com> wrote:
>
>> Data piles up in Cassandra without TTL. Is there a workaround for this
>> problem? Is there a way to specify
pingManager.
> We could add something similar as the ClusterBuilder for that though.
>
> Regards,
> Chesnay
>
>
> On 12.12.2016 16:15, Meghashyam Sandeep V wrote:
>
> Hi Till,
>
> Thanks for the information.
>
> 1. What do you mean by 'subtask', is i
ndra sink again.
>
> Cheers,
> Till
>
>
> On Fri, Dec 9, 2016 at 8:05 PM, Meghashyam Sandeep V <
> vr1meghash...@gmail.com> wrote:
>
>> Thanks a lot for the quick reply Shannon.
>>
>> 1. I will create a class that extends SinkFunction and write my
>&
n
> the position in or content in the stream. It's a necessary helper object,
> yes, but you don't need Flink to checkpoint it.
>
> You can still use the sinks provided with flink-connector-cassandra and
> customize the cluster building by passing your own ClusterBuilder into th
Hi there,
I have a flink streaming app where my source is Kafka and a custom sink to
Cassandra(I can't use standard C* sink that comes with flink as I have
customized auth to C*). I'm currently have the following:
messageStream
.rebalance()
.map( s-> {
return mapper.readValu
24 matches
Mail list logo
