Hey Joel, good questions As a first thought, my experience with customers in large corporate environments probably has me somewhat jaded :). You know it really shouldn't take 3 weeks to get ports opened on a load balancer, but, that really does happen. Coordination across those teams also can and should / does happen but I've noted that operators appreciate measures they can take that keep them out of more internal process.
1) Yes probably. After all we're really just checking what's returned from InetAddress and trusting that. The check is pretty lightweight. I think what you are getting at is that a security check that doesn't go all the way can be bad as it can engender a false sense sense of security and end up leaving the system more vulnerable to attack than if other, more standard, approaches are taken. This is a fair point. I'm not deep enough into network security to comment all that intelligently but I do think that reducing the exposure to say, IP spoofing on internal traffic vs free-for-all data consumption is a step in the right direction. 2) Yes they may have access to this, and it could be redundant. On customers that I interface with, operators typically get their root-level privileges through something like PowerBroker, so access to IPTables is not a given, and even if it's available typically does not fall within their realm of accepted responsibilities. Additionally, when I first got this request I suggested IPTables and was told that due to the difficulties and complexities of configuration and management (from their perspective) that it would not be an acceptable solution (also the "it's not in the corporate standard" line) I noted in the KIP that I look at this not only as a potential security measure by reducing attack vector size but also as a guard against human error. Hardcoded configs sometimes make their way all the way to production and this would help to limit that. You could argue that it might not be Kafka's responsibility to enforce this type of control, but there is precedence here with HDFS (dfs.hosts and dfs.hosts.exclude) and Flume (*https://issues.apache.org/jira/browse/FLUME-2189 <https://issues.apache.org/jira/browse/FLUME-2189>*). In short, I don't think that this supplants more robust security functionality but I do think it gives an additional (lightweight) control which would be useful. Security is about defense in depth, and this raises the bar a tad. Thanks Jeff On Tue, Mar 3, 2015 at 8:58 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > The proposal itself looks reasonable, but I have a couple of > questions as you made reference to "operators of the system; and > network team" in your wiki. > > - Are spoofing attacks a concern even with this in place? If so, it > would require some sort of internal ingress filtering which > presumably need cooperation with network teams right? > - Also, the operators of the (Kafka) system really should have access > to iptables on the Kafka brokers so wouldn't this feature be > effectively redundant? > > Thanks, > > Joel > > On Thu, Jan 22, 2015 at 01:50:41PM -0500, Joe Stein wrote: > > Hey Jeff, thanks for the patch and writing this up. > > > > I think the approach to explicitly deny and then set what is allowed or > > explicitly allow then deny specifics makes sense. Supporting CIDR > notation > > and ip4 and ip6 both good too. > > > > Waiting for KAFKA-1845 to get committed I think makes sense before > > reworking this anymore right now, yes. Andrii posted a patch yesterday > for > > it so hopefully in the next ~ week(s). > > > > Not sure what other folks think of this approach but whatever that is > would > > be good to have it in prior to reworking for the config def changes. > > > > /******************************************* > > Joe Stein > > Founder, Principal Consultant > > Big Data Open Source Security LLC > > http://www.stealth.ly > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > > ********************************************/ > > > > On Wed, Jan 21, 2015 at 8:47 PM, Jeff Holoman <jholo...@cloudera.com> > wrote: > > > > > Posted a KIP for IP Filtering: > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-7+-+Security+-+IP+Filtering > > > > > > Relevant JIRA: > > > https://issues.apache.org/jira/browse/KAFKA-1810 > > > > > > Appreciate any feedback. > > > > > > Thanks > > > > > > Jeff > > > > > -- Jeff Holoman Systems Engineer
