Re: Kafka bug ISR expands/shrinks "Count" JMX attribute doesn't work as expected

2018-04-20 Thread Stanislav Antic
I'm checking with prometheus using prometheus jmx exporter. When I shutdown one broker I don't have any line with: "Shrinking ISR from %s to %s" even though that ISR have been shrinked: Topic:test-topicPartitionCount:4ReplicationFactor:2 Configs: Topic: test-topic Part

Re: Commit in Kafka streams reduces throughput

2018-04-20 Thread Shivam Sharma
*Correction *- After removing commit my RPM increased to *40k/min* from *2k/min*. Thanks On Wed, Apr 18, 2018 at 11:22 PM, Guozhang Wang wrote: > Hello Shivam, > > I think your two questions can be answer as one :) > > Have just created this FAQ entry for you: > https://cwiki.apache.org/conflue

Using Kafka CLI without specifying the URLs every single time?

2018-04-20 Thread Horváth Péter Gergely
Hello All, I wondering if there is any way to avoid having to enter the host URLs for each Kafka CLI command you execute. This is kind of tedious as different CLI commands require specifying different servers (--broker-list, --bootstrap-server and --zookeeper); which is especially painful if the

Ordering guarantee after branching a stream

2018-04-20 Thread Botuck, Jacob (STL) - contr
If I call branch on a kStream, and I send record A into the trunkStream followed by record B. If records A and B go into separate branches. Is there any guarantee A will be processed before B?

Re: Using Kafka CLI without specifying the URLs every single time?

2018-04-20 Thread Martin Gainty
Hello Péter : etc/hosts PUBLIC_IP_ADDRESS FQDN SHORTNAME where shortname can be your shortened HostName (Nota Bene: be respectful of tabs, spaces and are important...best to make backup before modify) https://unix.stackexchange.com/questions/239920/how-to-set-the-fully-qualified-hostname-on

Re: Broker cannot start switch to Java9 - weird file system issue ?

2018-04-20 Thread Enrico Olivelli
It is a deliberate change in JDK code Just for reference see this discussion on nio-dev list on OpenJDK http://mail.openjdk.java.net/pipermail/nio-dev/2018-April/005008.html see https://bugs.openjdk.java.net/browse/JDK-8168628 Cheers Enrico 2018-03-05 14:29 GMT+01:00 Enrico Olivelli : > Wo

Re: Ordering guarantee after branching a stream

2018-04-20 Thread Matthias J. Sax
As long as no repartitioning happens yes. If one of both sub-streams is repartitioned, there is no guarantee. -Matthias On 4/20/18 4:31 PM, Botuck, Jacob (STL) - contr wrote: > If I call branch on a kStream, and I send record A into the trunkStream > followed by record B. If records A and B go

Re: Using Kafka CLI without specifying the URLs every single time?

2018-04-20 Thread Peter Bukowinski
One solution is to build wrapper scripts around the standard kafka scripts. You’d put your relevant cluster parameters (brokers, zookeepers) in a single config file (I like yaml), then your script would import that config file and pass the appropriate parameters to the kafka command. You could c

Re: Broker cannot start switch to Java9 - weird file system issue ?

2018-04-20 Thread Ismael Juma
Hi Enrico, Coincidentally, I saw your message to nio-dev and followed up there. Ismael On Fri, Apr 20, 2018 at 8:18 AM, Enrico Olivelli wrote: > It is a deliberate change in JDK code > > Just for reference see this discussion on nio-dev list on OpenJDK > http://mail.openjdk.java.net/pipermail

Re: Broker cannot start switch to Java9 - weird file system issue ?

2018-04-20 Thread Enrico Olivelli
Il ven 20 apr 2018, 20:24 Ismael Juma ha scritto: > Hi Enrico, > > Coincidentally, I saw your message to nio-dev and followed up there. > I think this is not a 'real' problem, you will notice the difference only if you have a lot of empty topics/partitions. In fact when you simply upgrade to jdk

Re: Broker cannot start switch to Java9 - weird file system issue ?

2018-04-20 Thread Ismael Juma
Hi Enrico, It is a real problem because it causes indexes to take a lot more disk space upfront. The sparsity is an important if people over partition, for example. Ismael On Fri, Apr 20, 2018 at 12:41 PM, Enrico Olivelli wrote: > Il ven 20 apr 2018, 20:24 Ismael Juma ha scritto: > > > Hi Enr

Kafka streams state directory - help

2018-04-20 Thread pradeep s
Hi, I am using kafka streams app connecting to confluent kafka cluster(10.2.1). Application is reading messages from a topic, performing a tranformation and pushing to output topic . There is no count or aggregation performed . Have following clarifications regarding state directory. *1)* Will the