Hi Guy, On Jan 6, 2013, at 11:11pm, Guy Doulberg wrote:
> Hi, > Thanks David, > > I am looking for a product (open source or not), something like Talend or > Pentaho that in which I can design the ETL (from and to kafka), and run the > the ETL in Storm/ IronCount or even maybe I can run it in Hadoop Map/Reduce. Interesting - we build ETLs on top of Hadoop using Cascading (open source workflow API), which has a lot of what it calls "Taps" for connecting to data sources and sinks. But I haven't heard of a Kafka Tap. Should be possible to implement, though. One issue is that Hadoop is batch oriented, so there's a bit of an impedance mismatch when you've got a streaming data source, but from experience it's possible to get that to work. -- Ken > The product should be complete and supports many connections to many data > sources and targets, In that sense if you know of a connection to Talend or > Pentaho it will be great. > > Thanks again. > , > > > On 01/07/2013 12:28 AM, David Arthur wrote: >> Storm has support for Kafka, if that's the sort of thing you're looking >> for. Maybe you could describe your use case a bit more? >> >> On Sunday, January 6, 2013, Guy Doulberg wrote: >> >>> Hi >>> >>> I am looking for an ETL tool that can connect to kafka, as a consumer and >>> as a producer, >>> >>> Have you heard of such a tool? >>> >>> Thanks >>> Guy >>> >>> > -------------------------- Ken Krugler +1 530-210-6378 http://www.scaleunlimited.com custom big data solutions & training Hadoop, Cascading, Cassandra & Solr
