Stream naming conventions?

2015-02-24 Thread Taylor Gautier
Hello all, Just wondering if those with a good amount of experience using Kafka in production with many streams have converged on any sort of naming convention. If so would you be willing to share? Thanks in advance, Taylor Gautier

Stream naming conventions?

2015-02-24 Thread Taylor Gautier
Hello all, Just wondering if those with a good amount of experience using Kafka in production with many streams have converged on any sort of naming convention.  If so would you be willing to share? Thanks in advance, Taylor

Re: Node-Kafka Client Review and Question

2013-04-24 Thread Taylor Gautier
This implementation is what I worked on while at Tagged, which was forked from Marcus' version, but I don't think it ever merged back to Marcus': https://github.com/tagged/node-kafka It was in production for about a year when I left Tagged about 6 months ago. I know that there were some internal

Re: Using Kafka for "data" messages

2013-06-13 Thread Taylor Gautier
I've been talking about this kind of architecture for years. As you said it's an EDA architecture. You might also want to have a look at Esper if you haven't already - it's a perfect complement to this strategy. At my last job I built a relatively low latency site wide pub sub system that showed

Re: Using Kafka for "data" messages

2013-06-13 Thread Taylor Gautier
Spot on. This one was of the areas that we had to workaround. Remember that there is a 1:1 relationship of topics to directories and most file systems don't like 10s of thousands of directories. We found on practice that 60k per machine was a practical limit using I believe EXT3FS On Thursday