Re: Consumer offsets partitions size much bigger than others

w > loud errors in the event you don't have enough brokers. > > -James > > > On Jul 18, 2017, at 8:44 AM, Luciano Afranllie > wrote: > > > > Hi > > > > One of our Kafka brokers was running out of disk space and when we > checked > > the fil

Consumer offsets partitions size much bigger than others

Hi One of our Kafka brokers was running out of disk space and when we checked the file size in the kafka log dir we observed the following $ du -h . --max-depth=2 | grep '__consumer_offsets' 4.0K./kafka-logs/__consumer_offsets-16 4.0K./kafka-logs/__consumer_offsets-40 35G ./kafka-logs

Re: Trying to understand design decision about producer ack and min.insync.replicas

ISR > settings. Instead, the ack setting just lets the producer control how > durable the write is before *that producer* can consider the write > "complete", i.e. before it gets an ack. > > -Ewen > > On Tue, Jan 24, 2017 at 12:46 PM, Luciano Afranllie < > listas.

Trying to understand design decision about producer ack and min.insync.replicas

Hi everybody I am trying to understand why Kafka let each individual producer, on a connection per connection basis, choose the tradeoff between availability and durability, honoring min.insync.replicas value only if producer uses ack=all. I mean, for a single topic, cluster administrators can't

Re: [GitHub] kafka pull request #1661: KAFKA-3987: Allow config of the hash algorithm use...

Hi Could you please tell me if this pull request can be merged and if so when can we expect this to happen? Thanks Luciano On Tue, Jul 26, 2016 at 9:01 AM, Luciano Afranllie wrote: > Hi > > Could somebody with commit permission review (and eventually merge) this > pull request?

[jira] [Updated] (KAFKA-3987) Allow configuration of the hash algorithm used by the LogCleaner's offset map

[ https://issues.apache.org/jira/browse/KAFKA-3987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luciano Afranllie updated KAFKA-3987: - Reviewer: Shikhar Bhushan Status: Patch Available (was: Open) > Al

Re: [GitHub] kafka pull request #1661: KAFKA-3987: Allow config of the hash algorithm use...

n the commit message: > > This closes #1661 > > > commit 2e7e507903c73740ca498405c5680a8c528ccda6 > Author: Luciano Afranllie > Date: 2016-07-25T14:39:59Z > > KAFKA-3987: Allow configuration of the hash algorithm used by the > LogCleaner's offset map > >

Re: Changing hash algorithm to LogCleaner offset map

it, makes sense to make the hash function customizable if there are > environments in which md5 usage is prevented. The approach you are > proposing sounds good to me. > On Sat, Jul 23, 2016 at 14:56 Luciano Afranllie > wrote: > > > Nothing wrong about using MD5 for that from

[jira] [Created] (KAFKA-3987) Allow configuration of the hash algorithm used by the LogCleaner's offset map

Luciano Afranllie created KAFKA-3987: Summary: Allow configuration of the hash algorithm used by the LogCleaner's offset map Key: KAFKA-3987 URL: https://issues.apache.org/jira/browse/KAFKA

Re: Changing hash algorithm to LogCleaner offset map

ion to use a FIPS-compliant hash function > for log compaction -- what are the security ramifications? > > On Fri, Jul 22, 2016 at 2:56 PM Luciano Afranllie < > listas.luaf...@gmail.com> > wrote: > > > A little bit of background first. > > > > We are trying t

Re: Changing hash algorithm to LogCleaner offset map

On Fri, Jul 22, 2016 at 5:58 PM, Luciano Afranllie wrote: > Hi > > We are evaluating to change the hash algorithm used by the SkimpyOffsetMap > used by the LogCleaner from MD5 to SHA-1. > > Besides the impact in performance (more memory, more cpu usage) is there > anything that may be impacted? > > Regards > Luciano >

Changing hash algorithm to LogCleaner offset map

Hi We are evaluating to change the hash algorithm used by the SkimpyOffsetMap used by the LogCleaner from MD5 to SHA-1. Besides the impact in performance (more memory, more cpu usage) is there anything that may be impacted? Regards Luciano

Re: Use Kafka 0.9.0.1 or 0.10.0.0 with Zookeeper 3.5

other wiki. Regards Luciano On Thu, Jun 30, 2016 at 8:42 AM, Luciano Afranllie wrote: > Thanks Flavio > > Your last statement is the key here. What we really need is to secure > connections between Kafka and zk and between zk nodes. Our understanding so > far is that zk 3.5 will allo

Re: Use Kafka 0.9.0.1 or 0.10.0.0 with Zookeeper 3.5

the same. You should be able to run a Kafka > cluster as is against a 3.5 ensemble, but you won't have a secure > connection to zk. > > -Flavio > > > On 29 Jun 2016, at 22:16, Luciano Afranllie > wrote: > > > > Hi > > > > I would like some ad

Use Kafka 0.9.0.1 or 0.10.0.0 with Zookeeper 3.5

Hi I would like some advice about what are the changes, at high level, required to use kafka 0.9.0.1 or 0.10.0.0 with Zookeeper 3.5.x (3.5.1-alpha for example) using TLS/SSL. How big are the changes required in Kafka in order to be able to do this? Regards Luciano

What about Audit feature?

Hi Kafka documentation mention Audit feature in section 6.6 but https://issues.apache.org/jira/browse/KAFKA-260 is resolved Won't Fix. Should this section of the documentation be removed? Regards Luciano

Status of multi-tenancy

Hi there We are interested in adding support for multiple tenants into Kafka and I reached to this thread http://grokbase.com/t/kafka/dev/154wsscrsk/adding-multi-tenancy-capabilities-to-kafka Could you please let me know the status of this proposal? Is this something we can move forward? Regar