Re: Changing hash algorithm to LogCleaner offset map

2016-07-24 Thread Luciano Afranllie
Thanks Shikhar. I have created KAFKA-3987 (https://issues.apache.org/jira/browse/KAFKA-3987). Could anybody please assign that issue to me? I could not do it. I have a patch ready and will open a pull request tomorrow. Regards On Sun, Jul 24, 2016 at 4:45 PM, Shikhar Bhushan wrote: > Got it,

Re: Changing hash algorithm to LogCleaner offset map

2016-07-24 Thread Shikhar Bhushan
Got it, makes sense to make the hash function customizable if there are environments in which md5 usage is prevented. The approach you are proposing sounds good to me. On Sat, Jul 23, 2016 at 14:56 Luciano Afranllie wrote: > Nothing wrong about using MD5 for that from FIPS point of view, but we w

Re: Changing hash algorithm to LogCleaner offset map

2016-07-23 Thread Luciano Afranllie
Nothing wrong about using MD5 for that from FIPS point of view, but we want to deploy with FIPS 140-2 mode enabled using only RSA security providers. With this settings it is not possible to use MD5. On Fri, Jul 22, 2016 at 8:49 PM, Shikhar Bhushan wrote: > Not sure I understand the motivation t

Re: Changing hash algorithm to LogCleaner offset map

2016-07-22 Thread Shikhar Bhushan
Not sure I understand the motivation to use a FIPS-compliant hash function for log compaction -- what are the security ramifications? On Fri, Jul 22, 2016 at 2:56 PM Luciano Afranllie wrote: > A little bit of background first. > > We are trying to make a deployment of Kafka that is FIPS 140-2 (

Re: Changing hash algorithm to LogCleaner offset map

2016-07-22 Thread Luciano Afranllie
A little bit of background first. We are trying to make a deployment of Kafka that is FIPS 140-2 ( https://en.wikipedia.org/wiki/FIPS_140-2) complaint and one of the requirements is not to use MD5. As far as we could see, Kafka is using MD5 only to hash message keys in a offset map (SkimpyOffsetM

Changing hash algorithm to LogCleaner offset map

2016-07-22 Thread Luciano Afranllie
Hi We are evaluating to change the hash algorithm used by the SkimpyOffsetMap used by the LogCleaner from MD5 to SHA-1. Besides the impact in performance (more memory, more cpu usage) is there anything that may be impacted? Regards Luciano