Congrats!
~ Joe Stein
On Fri, Jun 9, 2017 at 6:49 PM, Neha Narkhede wrote:
> Well deserved. Congratulations Damian!
>
> On Fri, Jun 9, 2017 at 1:34 PM Guozhang Wang wrote:
>
> > Hello all,
> >
> >
> > The PMC of Apache Kafka is pleased to announce that w
nting
and communicating their semantic version strategy.
If JDK8 is so much desired then jump to 0.11 and only do bug fixes on the
0.10 release (which should be rigorous and not forceful to make folks
upgrade unnecessarily to get such improvements).
My 0.2824152382 cents.
Regards,
~ Joe Stein
O
Topology
Looking forward to continued contributes, if you haven't tried out Heron
yet you should.
Kafka is the wind to make Heron fly!
/******
Joe Stein
Elodina Inc
http://www.elodina.net
**/
If you use Go you can use https://github.com/sclasen/event-shuttle which is
nice choice in some cases because footprint, it uses boltdb which is like
leveldb which is like embedded k/v ok
NiFi is cool too https://nifi.apache.org/
So is bruce https://github.com/ifwe/bruce
those are more out of th
There is one being worked on here
https://github.com/tuplejump/kafka-connect-cassandra/
On Wed, Apr 13, 2016 at 5:21 PM, Kaz Chehresa wrote:
> Has anyone tried implementing or know whether it's possible to implement a
> "Kafka Connect" source connector for apache Cassandra, similar to its
> JDBC
ok, you should be gt2g
~ Joe Stein
- - - - - - - - - - - - - - - - - - -
[image: Logo-Black.jpg]
http://www.elodina.net
http://www.stealth.ly
- - - - - - - - - - - - - - - - - - -
On Mon, Nov 30, 2015 at 1:05 PM, Andrew Schofield <
andrew_schofi...@uk.ibm.com> wrote:
>
ed or anything lots of info in there too for folks that are
helpful.
I think more services that folks know have kafka behind it gives it more
reliability that it can do what the expectation for it are to-do.
~ Joe Stein
On Mon, Nov 30, 2015 at 12:39 PM, Andrew Schofield <
andrew_schofi...@uk.ibm.co
Thanks for everyone that contributed to this release! It has been a long
time in the works with some really great new additions for folks waiting
with excitement of the new consumer, security and connect (copycat) and
everything else baked in.
Thanks again!
~ Joe Stein
They should all be on the user groups section of the confluence page
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations
for which there were video. It might need some curating but that is where
it has been going so far.
~ Joe Stein
On Tue, Nov 3, 2015 at 4:48 PM
best support for integrated
systems... I suspect that integrated system will support Kafka too :)
The Hadoop community has spent A LOT of time since 0.8 making all of that
ecosystem more cohesive within that community. I urge support for it always.
~ Joe Stein
- - - - - - - - - - - - - - - - - - -
ntral so you will need to download into a local mvn and
use 0.8.2.2 producer. I don't know it if will fix your problem but if you
can reproduce it and see and if not post to a JIRA that would be great.
Thanks!
~ Joe Stein
- - - - - - - - - - - - - - - - - - -
[image: Logo-Black.j
Is this something also that should be in 0.8.2.2
https://issues.apache.org/jira/browse/KAFKA-2421 wasn't sure what the lz4
usage has been seems to break on some jdks.
+1 on the snappy fixes
~ Joe Stein
On Fri, Aug 14, 2015 at 5:39 PM, Guozhang Wang wrote:
> +1 for both KAFKA-2189
available in contrib repo we can link to.
Other thoughts/comments happy to catch those and update the KIP.
Thanks!
~ Joe Stein
- - - - - - - - - - - - - - - - - - -
[image: Logo-Black.jpg]
http://www.elodina.net
http://www.stealth.ly
- - - - - - - - - - - - - - - - - - -
On Sun, Aug 9
yup, my bad I really thought we got that one in but it is on trunk so 0.8.3
it is
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Fri, Jun 19, 2015 at 8:52 AM, Todd Palino wrote:
> I don't think this got changed until after
What version of Kafka are you using? This was changed to debug level in
0.8.2.
~ Joestein
On Jun 18, 2015 10:39 PM, "bit1...@163.com" wrote:
> Hi,
> I have started the kafka server as a backgroud process, however, the
> following INFO log appears on the console very 10 seconds.
> Looks it is not
+1 to using exhibitor. Besides managing the ensemble it also helps with
backups and zk log cleanup (which if you don't do your machine will run out
of space).
~ Joestein
On Jun 17, 2015 9:44 AM, "Dillian Murphey" wrote:
> supervisord is pretty easy to use. Netflix Exhibitor will manage this all
If your running kafka on mesos you should use the framework
https://github.com/mesos/kafka as the scheduler is better opinionated than
generic launch on marathon.
~ Joestein
On Jun 11, 2015 10:26 AM, "Domen Pogacnik" <
domen.pogac...@rocket-internet.de> wrote:
> @Jeff Schroeder:
>
> I’m trying to
discussion again maybe in a few weeks after folks have
had a chance to test and try out all what is done/almost done.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Fri, May 29, 2015 at 10:04 AM, Kuchibhotla, Murthy <
venkata.murthy.kuchi
Try out bruce
https://github.com/ifwe/bruce it's a daemon listening socket producer, does
exactly what you are looking for I think.
~ Joestein
On May 19, 2015 7:05 AM, "clay teahouse" wrote:
> Thanks Magnus. I'll take a look at n2kafka. I have many data sources
> sending data to kafka and I don'
7;t have a enough
replica to survive data loss guarantees. min.isr=2 min.sir=3 //depends on
data
Also take a look at
https://github.com/stealthly/go_kafka_client/tree/master/mirrormaker it
might be helpful for what you are looking for.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.
thub.com/wvanbergen/kafka - that speaks the consumergroup API
> and
> accomplishes what I was looking for and I assume is implemented by keeping
> track of memberships w/ Zookeeper.
>
> Thank you for the information - it really helped clear up what I failing to
> understand with kafka.
&g
pache.org/learn/documentation/0.9/comparisons/introduction.html
which has a Mesos patch https://issues.apache.org/jira/browse/SAMZA-375 and
built in YARN support.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Wed, Apr 29, 2015 at 8:56 AM, Davi
Nice!
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Sun, Mar 22, 2015 at 12:03 AM, Joseph Lawson wrote:
> Hi everyone,
>
>
> I updated jruby-kafka to version 1.4.0 which uses Kafka 0.8.2.1 and has a
> new interface
https://kafka.apache.org/documentation.html#basic_ops_cluster_expansion
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Fri, Mar 13, 2015 at 3:05 PM, sunil kalva wrote:
> Joe
>
> "Well, I know it is semantic but right n
ling today with AWS CloudFormation and internal systems they
built too.
So, it can be done... you just have todo it.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Fri, Mar 13, 2015 at 12:39 PM, Stevo Slavić wrote:
> OK, thanks
to balance
partitions when increasing/decreasing the size of the cluster and some more
goodies too.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Fri, Mar 13, 2015 at 8:43 AM, Stevo Slavić wrote:
> Hello Apache Kafka community,
Thanks Jun for getting this release out the door and everyone that
contributed to the work in 0.8.2.1, awesome!
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Mon, Mar 9, 2015 at 2:12 PM, Jun Rao wrote:
> The following are
requirements or were looking for something
higher level like Sqoop or NiFi (neither which I think support both yet) or
something.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Tue, Mar 10, 2015 at 3:14 AM, Vadim Keylis wrote:
> Good even
oesn't provide the information everyone needs fluidly enough.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Wed, Mar 4, 2015 at 12:44 PM, Evan Huus wrote:
> Hey all, it seems that 0.8.2 has added a handful more errors to the
Ok, lets fix the transient test failure on trunk agreed not a blocker.
+1 quick start passed, verified artifacts, updates in scala
https://github.com/stealthly/scala-kafka/tree/0.8.2.1 and go
https://github.com/stealthly/go_kafka_client/tree/0.8.2.1 look good
~ Joe Stein
l(JUnit3Suite.scala:149)
at
kafka.api.ProducerFailureHandlingTest.testNotEnoughReplicasAfterBrokerShutdown(ProducerFailureHandlingTest.scala:355)
This happens to me all the time on a few different machines.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Mon, Mar 2, 2015 at 7:3
It can be done, sure. We built a prototype a while back
https://github.com/stealthly/f2k though I can't say I have bumped into a
use case where the tradeoffs worked to do it. Chunking the file in each
message and reconstructing it is going to be an overhead or your going to
block on waiting to writ
Sounds like https://issues.apache.org/jira/browse/KAFKA-1788 maybe
On Feb 24, 2015 2:28 PM, "Scott Chapman" wrote:
> Yea, however I don't get async behavior. When kafka is down the log blocks,
> which is kinda nasty to my app.
>
> On Tue Feb 24 2015 at 2:27:09 PM Joe S
PM, Joe Stein wrote:
> and kafka too :)
>
> On Tue, Feb 24, 2015 at 2:21 PM, Joe Stein wrote:
>
>> are you including
>> https://github.com/stealthly/scala-kafka/blob/master/build.gradle#L122
>> in your project?
>>
>> ~ Joe Stein
>> - - - - - - - -
Producer type isn't needed anymore with the new producer so in the the
logger properties just leave that out in 0.8.2 and it should work.
On Tue, Feb 24, 2015 at 2:24 PM, Joe Stein wrote:
> Interesting, looks like a breaking change from 0.8.1
> https://github.com/apache/kafka/blob
are you including
https://github.com/stealthly/scala-kafka/blob/master/build.gradle#L122 in
your project?
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Tue, Feb 24, 2015 at 2:02 PM, Scott Chapman wrote:
> Yea, when I try to set t
and kafka too :)
On Tue, Feb 24, 2015 at 2:21 PM, Joe Stein wrote:
> are you including
> https://github.com/stealthly/scala-kafka/blob/master/build.gradle#L122 in
> your project?
>
> ~ Joe Stein
> - - - - - - - - - - - - - - - - -
>
>
and changed it as things came up... e.g. setSource(),
setTags() blah blah...
Paul Otto has an open source version
https://github.com/potto007/kafka-appender-layout that you could try out
too that he built to tackle some of the layout things.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http
Jun,
Can we also add https://issues.apache.org/jira/browse/KAFKA-1724 to the
next RC please?
Thanks!
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Sun, Feb 22, 2015 at 11:59 AM, Jun Rao wrote:
> We identified at least one m
-1729 please.
+1 (binding)
~ Joe Stein
On Fri, Feb 20, 2015 at 9:08 PM, ted won wrote:
> +1
>
> On Friday, February 20, 2015, Guozhang Wang wrote:
>
> > +1 binding.
> >
> > Checked the md5, and quick start.
> >
> > Some minor comments:
> >
> &
We pushed first bits for the kafka framework we are working on to
https://github.com/mesos/kafka.
We have been using Aurora and Marathon to run Kafka on Mesos but are
cutting over to a framework approach as described in the ticket over the
next 5 weeks.
~ Joe Stein
in scala
https://github.com/stealthly/scala-kafka/blob/master/src/test/scala/KafkaSpec.scala#L146-L168
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Sat, Feb 21, 2015 at 8:10 AM, Cdhar B wrote:
> Hi,
>
> Could you pleas
The Apache project doesn't have a web console for kafka.
Have you taken a look at https://github.com/yahoo/kafka-manager as of yet?
I haven't myself hoping to get sometime tonight/this weekend to-do so.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whypartitionleadersmigratethemselvessometimes
?
~ Joe Stein
- - - - - - - - - - - - - - - - -
http://www.stealth.ly
- - - - - - - - - - - - - - - - -
On Fri, Feb 20, 2015 at 7:19 PM, Sa Li wrote:
> Hi, All
>
> My dev cl
Lets have a KIP for removing JDK 6 support (on another thread) as it would
make for a good discussion and something we should be able to come to
closure on as a dependency for this and other patches that would favor it
happening.
~ Joe Stein
- - - - - - - - - - - - - - - - -
http
Cool!!!
Maybe on the eco system confluence page we should have a packaging and/or
distro section
~ Joestein
On Wed, Feb 11, 2015 at 1:34 PM, David Morales wrote:
> Regarding RPM/DEB packages, we, at stratio.com, have a public repository
> which includes RPM and dev packages
>
> http://repo
Thanks Harsha, looks good so far. How were you thinking of running
the KerberosTicketManager as a standalone process or like controller or is
it a layer of code that does the plumbing pieces everywhere?
~ Joestein
On Wed, Feb 11, 2015 at 12:18 PM, Harsha wrote:
> Hi,
> Here is the initial p
One good canary is to have a topic you loop through every partition and
write a timestamp on every broker for n in 1..P; do; echo
$(datetime)>kafkacat -b host:port -t ops -p n done and trap any errors and
output accordingly.
You can get fancy and start validating reading and stuff but it is a cana
know
what their release plan is though.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
**
You have to delete the znode from zookeeper for your consumer group
https://github.com/apache/kafka/blob/0.8.1/core/src/main/scala/kafka/consumer/ConsoleConsumer.scala#L181
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
Ankur,
There is more from papers and presentations you can check out too
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+papers+and+presentations
if you haven't already.
- Joestein
On Thu, Feb 5, 2015 at 12:57 PM, Conikee wrote:
> Michael Noll's blog posting might serve your purpose as
not as so much sync vs async but my
crystal ball just back from the shop so well see =8^)
/*******
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/al
too (same line --partitions).
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
/
Huzzah!
Thanks Jun for preparing the release candidates and getting this out to the
community.
- Joe Stein
On Mon, Feb 2, 2015 at 2:27 PM, Jun Rao wrote:
> The following are the results of the votes.
>
> +1 binding = 3 votes
> +1 non-binding = 1 votes
> -1 = 0 votes
> 0 = 0
+1 (binding)
verified signatures, ran quick start, tests all passed.
- Joe Stein
On Fri, Jan 30, 2015 at 12:04 PM, Jun Rao wrote:
> This is a reminder that the vote will close tomorrow night. Please test
> RC3 out and vote before the deadline.
>
> Thanks,
>
> Jun
>
>
rs is taking a look at that now unless someone is already working on
that? We are using the partition static assignment in the new consumer and
just using Mesos for handling re-balance for us. When he gets further along
and if it makes sense we will shoot a KIP around people can chat about it
on dev.
Matve wr should add to the documentation experimental so folks that don't
know understand.
/*******
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingsh
topic deletion,
> compaction and random testing
>
> On Mon, Jan 26, 2015 at 6:14 AM, Neha Narkhede wrote:
>
>> +1 (binding)
>> Verified keys, quick start, unit tests.
>>
>> On Sat, Jan 24, 2015 at 4:26 PM, Joe Stein wrote:
>>
>> > That makes sense, tha
th the old clients and the
> conclusion is that serialization is something you basically have to think
> about to use Kafka and trying to guess just makes things worse.
>
> -Jay
>
> On Sat, Jan 24, 2015 at 2:51 PM, Joe Stein wrote:
>
>> Maybe. I think the StringSerialzer
hat exception will make them ask that
> question right?
>
> -Jay
>
> On Fri, Jan 23, 2015 at 9:06 PM, Joe Stein wrote:
>
>> Should value.serializer in the new java producer be defaulted to
>> Array[Byte] ?
>>
>> I was working on testing some upgrade paths
Should value.serializer in the new java producer be defaulted to
Array[Byte] ?
I was working on testing some upgrade paths and got this
! return exception in callback when buffer cannot accept message
ConfigException: Missing required configuration "value.serializer"
which has no defau
The new consumer is scheduled for 0.9.0.
Currently Kafka release candidate 2 for 0.8.2.0 is being voted on.
There is an in progress patch to the new consumer that you can try out
https://issues.apache.org/jira/browse/KAFKA-1760
/***
Joe Stein
Founder
Vishal,
Does this error happen every time you are sending? Or just the first time?
Joe Stein
On Thu, Jan 22, 2015 at 4:33 AM, wrote:
> Hi,
> Let me overview on the issue that I am facing on producing message in
> Kafka:
> I have horthonworks HDP-2.1 installed, along that we h
he list of kafka brokers (through a metadatarequest) and their current
ownership. More on that here
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-Partitioningandbootstrapping
if you are interested.
/******
Sounds like you are bumping into this
https://issues.apache.org/jira/browse/KAFKA-1367
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.
This is a stdin/stdout producer/consumer that works great for (what I
think) you are trying to-do https://github.com/edenhill/kafkacat
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter
*fetch.message.max.bytes* in your consumers properties (whoever that might
be configured for your specific consumer being used) so that they can read
that data otherwise you won't see messages downstream.
/***
Joe Stein
Founder, Principal Consultant
It works ok in gradle but fails if your using maven.
taking a look at the patch you uploaded now.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.
either way. The
above is less lint in zk long term.
It is all just 1s and 0s and just a matter of how many you put together
yourself vs take out of the box that are given too you =8^)
/***********
Joe Stein
Founder, Principal Consultant
Big Data Open Source Secur
-statsd-metrics2 and sending to
https://github.com/kamon-io/docker-grafana-graphite you can quickly chart
and see everything going on.
There are also software as a service options too.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security
try out and get feedback so that as we continue to
stabilize the existing deployments we can build in more usage scenarios.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <h
I think that is a change of behavior that organizations may get burned on.
Right now there is no delete data feature. If an operations teams upgrades
to 0.8.2 and someone decides to delete a topic then there will be data
loss. The organization may not have wanted that to happen. I would argue to
no
consuming with Kafka through Sarama.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
/
/infrastructure/aws/bastion)
and supervisor (https://github.com/stealthly/minotaur/tree/master/supervisor)
and after that it is really nice and easy.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter
this help, thanks!
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
/
On Fri, Jan 9
You need to export the JMX_PORT for kafka to use on startup before starting
up
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshad
and nothing breaks =8^)
/*******
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
/
On Wed, Jan 7, 2015 at 1:04 PM,
counting the message from each partition using
https://github.com/edenhill/kafkacat and pipe to wc -l it makes for a nice
simple sanity check to where the problem might be.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http
for shared memory file:
/tmp/hsperfdata_root/19721
I don't think this is Kafka related though so what we are talking about
partition and retention is not applicable, even though important.
/*******
Joe Stein
Founder, Principal Consultant
Big
think we should treat the version field not just to stop from breaking
the wire protocol calls but also as a "feature flag" preserving upgrades
and multiple pathways.
I updated the JIRA for the feature flag needs for OffsetFetch and
OffsetCommit too.
/******
t
in these scenarios.
/*******
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
/
On Fri, Jan 2, 2015 at 8:25 PM, Mukes
which is the default likely for your OS.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
**
you should probably have it be consistent on each node.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshad
That is because your logs are in /tmp which you can change by
setting log.dirs to something else.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.
sumer/ConsoleConsumer.scala#L196
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthin
as such... bursty traffic (especially with batched
messages) can upset this, yup.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshad
works
right from the repo better without having to-do much local for others so
they don't bump into this and then work around it locally by installing
protobuf.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
Thunder, can you add that to
https://cwiki.apache.org/confluence/display/KAFKA/Clients#Clients-.net
didn't know it existed but cool that it uses Rx
On Mon, Dec 22, 2014 at 12:52 PM, Thunder Stumpges
wrote:
> Hi there,
>
> We looked at both of these a while back and ended up writing our own (
> h
Another option is a HTTP wrapper around the actual producer and doing a
HTTP POST from C# to a REST server e.g.
https://github.com/stealthly/dropwizard-kafka-http which I know folks have
done successfully.
/***
Joe Stein
Founder, Principal Consultant
Big
commit your offset in batch post processing. Then you only need more
partitions to fetch batches to process in parallel.
Regards, Joestein
On Fri, Dec 19, 2014 at 12:01 PM, Joe Stein wrote:
>
> see some comments inline
>
> On Fri, Dec 19, 2014 at 11:30 AM, Achanta Vamsi Subhash &l
see some comments inline
On Fri, Dec 19, 2014 at 11:30 AM, Achanta Vamsi Subhash <
achanta.va...@flipkart.com> wrote:
>
> We require:
> - many topics
> - ordering of messages for every topic
>
Ordering is only on a per partition basis so you might have to pick a
partition key that makes sense for
I would suggest to use the new client java producer in 0.8.2-beta
http://kafka.apache.org/082/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html
it handles the case you brought up (among lots of other goodies).
/***
Joe Stein
Founder, Principal
Nice!
On Mon, Dec 15, 2014 at 11:41 AM, Roger Hoover
wrote:
>
> Joseph,
>
> That's great! Thank you for writing that plugin.
>
> Cheers,
>
> Roger
>
> On Mon, Dec 15, 2014 at 7:24 AM, Joseph Lawson
> wrote:
> >
> > ?Kafka made some headlines with Logstash announcing their latest version
> > bet
?
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
/
On Sun, Dec 14, 2014 at 8:22 PM, Koert Kuipers wrote:
>
> hello
program, or you can use
as reference and rewrite it in Java.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshad
r down, and the topic replica is set to 1, the
> whole system is still available and the data would go to other partitions
> without loss.
>
>
> THANKS again.
>
> On Thu, Dec 11, 2014 at 12:37 AM, Joe Stein wrote:
>
> > If you want no data loss then you need to se
Hi, we open sourced a new Go Kafka Client
http://github.com/stealthly/go_kafka_client some more info on blog post
http://allthingshadoop.com/2014/12/10/making-big-data-go/ for those working
with Go or looking to get into Go and Apache Kafka.
/***
Joe Stein
no messages will be lost as long as at least one in sync
replica remains.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshad
.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
/
On Tue, Dec 9, 2014 at 2:54 AM, ashendra bansal
wrote:
> Hi,
>
>
Cloudera Manager utilizes the chroot znode structures so you should connect
with --zookeeper localhost:2181/kafka
Or whatever value CM has set for the chroot path of your installation.
/***
Joe Stein
Founder, Principal Consultant
Big Data Open Source
1 - 100 of 387 matches
Mail list logo
