Yes, and the only way to enforce it is by restricting ZK access to brokers
only. If ZK access is available to users, they can get around the broker
config proposal as well. And then the question is about benefit versus
cost. We can have that discussion when the KIP is proposed.
Ismael
On Wed, May
Certainly. I think it is reasonable to create a separate KIP to enforce the
topic creation policy. After all the administrator needs a guarantee that
the policy that they have specified in the broker will be enforce --
otherwise the feature doesn't seem complete.
On Tue, May 30, 2017 at 4:45 PM, I
I am not sure if the additional complexity in the Controller is worth it
for this use case. It seems like it would be better to swap the tools to
use AdminClient and then restrict access to ZK (via ACLs and/or network
segmentation). Either way, that proposal should be done via a separate KIP
as KIP
On Tue, May 30, 2017 at 4:26 PM, Colin McCabe wrote:
> On Tue, May 30, 2017, at 15:55, Dong Lin wrote:
> > Hey Colin,
> >
> > I think one big advantage of the broker side config is that it can not be
> > ignored by the malicious client, right?
>
> Hi Dong,
>
> The scenario I was thinking of is wh
On Tue, May 30, 2017, at 15:55, Dong Lin wrote:
> Hey Colin,
>
> I think one big advantage of the broker side config is that it can not be
> ignored by the malicious client, right?
Hi Dong,
The scenario I was thinking of is where a malicious client communicates
directly with ZooKeeper, bypassing
@Colin checkout kafka.admin.ZkSecurityMigrator. This is what I meant in my
earlier comment on "acl off zookeeper"
On Tue, May 30, 2017 at 3:39 PM Colin McCabe wrote:
> It seems like, to make it really secure, we need the enforcement to be
> done at the ZooKepeer level. Any broker or client-side
Hey Colin,
I think one big advantage of the broker side config is that it can not be
ignored by the malicious client, right?
Thanks,
Dong
On Tue, May 30, 2017 at 3:53 PM, Dong Lin wrote:
> Do we have an old version of bin/kafka-topics.sh which creates topic via
> ZK and still allows user to ac
Do we have an old version of bin/kafka-topics.sh which creates topic via ZK
and still allows user to access ZK with ACL? Another concern is that some
user may not have ACL service deployed in their cluster. If neither of
these is issue, then I would prefer the zookeeper approach instead of
adding
It seems like, to make it really secure, we need the enforcement to be
done at the ZooKepeer level. Any broker or client-side configuration
can just be ignored by a malicious client. Do we have documentation or
code that configures ZK to prevent unprivileged users from modifying the
topic configu
Just for completeness, I think one option is to:
1. Upgrade broker-side auto topic creation to send CreateTopicsRequest.
This is just to unify the topic creation flow and policy enforcement.
2. Migrate all clients away from topic creation from zookeeper and instead
send CreateTopicsRequest
3. ACL o
Hey Ismael,
I agree that it makes sense not to cover ZK-based topic creation with the
topic creation policy and limit ZK access to brokers only going forward. My
point is that we need a way to disable ZK-based topic creation so that all
topic creation goes through the topic creation policy as spec
Hi Dong,
No, ZK-based topic creation doesn't go through the policy since it doesn't
go through the broker. Given that, I am not sure how the broker config
would work. Can you please elaborate? It seems like the way forward is to
limit ZK access to brokers only.
Ismael
On Tue, May 30, 2017 at 10:
Hey Ismael,
Thanks for the KIP. This is definitely useful.
Does the KIP apply the topic creation policy to ZK-based topic creation? If
not, which seems to be the case from my understanding, should we have a new
broker config to disable ZK-based topic creation? This seems necessary to
prevent user
Got it. Thanks, Ismael.
On Mon, Jan 9, 2017 at 10:42 AM, Ismael Juma wrote:
> Hi Roger,
>
> That's a good question. The server defaults are passed via the `configure`
> method of the `Configurable` interface that is implemented by
> `CreateTopicPolicy`. I'll mention this explicitly in the KIP.
Hi Roger,
That's a good question. The server defaults are passed via the `configure`
method of the `Configurable` interface that is implemented by
`CreateTopicPolicy`. I'll mention this explicitly in the KIP.
Ismael
On Mon, Jan 9, 2017 at 6:04 PM, Roger Hoover wrote:
> This is great. Thanks,
This is great. Thanks, Ismael.
One question. When TopicDetails are passed to the policy implementation,
would the server defaults already have been merged? If not, I think the
policy also needs access to the server defaults.
Cheers,
Roger
On Fri, Jan 6, 2017 at 9:26 AM, Ismael Juma wrote:
Thanks for the review Jun. Yes, that's a good point, I have updated the KIP.
Ismael
On Fri, Jan 6, 2017 at 5:15 PM, Jun Rao wrote:
> Hi, Ismael,
>
> Thanks for the KIP. Looks reasonable to me. To be consistent with the
> pattern used in other pluggable interfaces, we probably should make the ne
Hi, Ismael,
Thanks for the KIP. Looks reasonable to me. To be consistent with the
pattern used in other pluggable interfaces, we probably should make the new
interface configurable and closable?
Jun
On Fri, Jan 6, 2017 at 4:16 AM, Ismael Juma wrote:
> Thanks Dan and Colin for the feedback. I u
Thanks Dan and Colin for the feedback. I updated the KIP to include the
addition of a validation mode. Since we need to bump the protocol version
for that, I also added an error message per topic to the response. I had
the latter as "Future Work", but I actually felt that it should be in the
first
Yeah, I agree... having a validation mode would be nice. We should be
explicit that passing validation doesn't 100% guarantee that a
subsequent call to create the topic will succeed, though. There is an
obvious race condition there-- for example, with a plugin which consults
some external authent
it would be nice to have a dry-run or validate ability added to this kip.
since we are offloading validation to a 3rd party implementor a random user
can't know a priori (based solely on kafka configs) whether a call should
succeed without actually creating the topic.
a similar case is in connect
Hi all,
We've posted "KIP-108: Create Topic Policy" for discussion:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-108%3A+Create+Topic+Policy
Please take a look. Your feedback is appreciated.
Thanks,
Ismael
22 matches
Mail list logo
