Re: map reduce Outputformat KafkaOutputFormat

[Joe Stein]

Abraham,

The last stable version of that is
https://github.com/apache/kafka/tree/0.8.1/contrib/hadoop-producer however;
there isn't a lot of support for it.  Folks have moved towards
https://github.com/linkedin/camus and/or
http://www.cloudera.com/content/cloudera/en/documentation/cdh5/latest/CDH5-Installation-Guide/cdh5ig_flume_package_install.html
for Kafka <=> Hadoop.  You can also use Spark read from kafka create RDD
save in HDFS or the latest Storm HDFS connector as a bolt from the Kafka
Spout too.  Sorta depends on your use case.

/***
 Joe Stein
 Founder, Principal Consultant
 Big Data Open Source Security LLC
 http://www.stealth.ly
 Twitter: @allthingshadoop 
/

On Wed, Oct 1, 2014 at 2:38 PM, Abraham Jacob  wrote:

> Hi All,
>
> After a map reduce computation I would like to send the results out to the
> a Kafka queue. Is there a bridge available that will let me publish the
> output to a Kafka queue.
>
> Similar to the TextOutputFormat... maybe a KafkaOutputFormat...
>
> I found one source
>
> https://github.com/kafka-dev/kafka/tree/master/contrib/hadoop-producer/src/main/java/kafka/bridge/hadoop
>
> But was still wondering about experience from others regarding this topic.
>
>
> Thanks,
> -abe
>
>
> --
> ~
>

Changing number of partitions after topic creation tool

[jova]

I was reading online that a tool was coming to change the number of
partitions of a topic after it has been created.  Is this tool available
yet?


-- 
Jovaughn Lockridge

{developer : Github  }
{professional : LinkedIn 
 }
{social : twitter  }

Re: Changing number of partitions after topic creation tool

[Neha Narkhede]

You can do that with the topics command. Refer to the docs here -
http://kafka.apache.org/documentation.html#basic_ops_add_topic

On Sat, Oct 4, 2014 at 4:29 PM, jova  wrote:

> I was reading online that a tool was coming to change the number of
> partitions of a topic after it has been created.  Is this tool available
> yet?
>
>
> --
> Jovaughn Lockridge
>
> {developer : Github  }
> {professional : LinkedIn <
> https://www.linkedin.com/profile/view?id=20227395>
>  }
> {social : twitter  }
>

Re: Changing number of partitions after topic creation tool

[jova]

Thanks.

On Sat, Oct 4, 2014 at 7:36 PM, Neha Narkhede 
wrote:

> You can do that with the topics command. Refer to the docs here -
> http://kafka.apache.org/documentation.html#basic_ops_add_topic
>
> On Sat, Oct 4, 2014 at 4:29 PM, jova  wrote:
>
> > I was reading online that a tool was coming to change the number of
> > partitions of a topic after it has been created.  Is this tool available
> > yet?
> >
> >
> > --
> > Jovaughn Lockridge
> >
> > {developer : Github  }
> > {professional : LinkedIn <
> > https://www.linkedin.com/profile/view?id=20227395>
> >  }
> > {social : twitter  }
> >
>



-- 
Jovaughn Lockridge

{developer : Github  }
{professional : LinkedIn 
 }
{social : twitter  }