It must lead to One - Re: Kafka algebra

2014-07-19 Thread Robert Withers
If a 3d Beatrix models the set of 4d spaces, then a 4d snoopy would describe a set of 5d spaces. If you treat the elements of the matrix as a complex sub-matrix, there is no reason it couldn't be a higher dimension super-matrix. Then a higher dimension matrix could be described by a lower dime

Kafka algebra

2014-07-19 Thread Rob Withers
I think you guys have created a system that demonstrates a new field of mathematics, unless I am missed something in my research on the topic. If we define the system as being an N-dimensional space of R-dimensional subspaces, does this mean we need to use a real 3D matrix to model the algebra

Re: Improving the Kafka client ecosystem

2014-07-19 Thread Mark Roberts
Hi all, As a client engineer on the python client, I would really appreciate a separate mailing list for client implementation discussion and a language agnostic test suite. What might also be really useful is an enumerated list of error conditions and the expected behavior to come out of them.

Re: Improving the Kafka client ecosystem

2014-07-19 Thread Jay Kreps
Hey Philip, Yeah I think we have actually done pretty good at getting reasonably solid clients in a bunch of languages. I just think it is an important area. The architecture design patterns idea is fantastic. That would be a great thing to do. -Jay On Fri, Jul 18, 2014 at 11:46 PM, Philip O'

Re: Performance/Stress tools

2014-07-19 Thread Steve Morin
Otis, Yes this would work for Kafka because it's using to launch containers to generate load for performance testing. It also works in standalone mode to run on a single machine. We currently are updating it to make it easier to use and upgrading it's ability to collect statistics in distribute

Re: New Consumer Design

2014-07-19 Thread Robert Withers
Lock is a bad way to say it; a barrier is better. I don't think what I am saying is even a barrier, since the rebalance would just need to recompute a rebalance schedule and submit it. The only processing delay is to allow a soft remove to let the client cleanup, before you turn on the new guy

Re: Performance/Stress tools

2014-07-19 Thread Otis Gospodnetic
Would this work for Kafka, Steve, considering Kafka doesn't use Yarn? Dayo - we developed https://github.com/sematext/ActionGenerator to feed events to things like Elasticsearch, Solr, and MongoDB. We'd gladly take a pull request for Kafka. SPM for Kafka

Switch Apache logo URL to HTTPS

2014-07-19 Thread Marcin ZajÄ…czkowski
Hi, Reading Kafka webpage/documentation through HTTPS (e.g. https://kafka.apache.org/documentation.html) I spotted that it generates warning (in Firefox and Chromium) about a reference to unencrypted resources. In fact the Apache logo is always load using HTTP - http://www.apache.org/images/feathe

Re: Improving the Kafka client ecosystem

2014-07-19 Thread Timothy Chen
The certified client test suite really will benefit all the client developers, as writing a Kafka client often is not just talking protocol but to be able to handle correctly all the cases, errors and situations, but also performance. From my experience writing a C# client definitely feel that a