I have created 2 branches in my repository to implement the Trie changes.
-
https://github.com/Claudenw/kafka/compare/trunk...StandardAuthorizer_refactor
refactors the authorizer code to make it easy to implement a different
Authorizor data block that plugs into the StandardAuthorizo
>
> Proposed Changes
> This KIP suggests to support *MATCH* resource pattern type when creating
> a new kafka acl.
>
I do not think we need the MATCH support within the Authorizer as noted in
my earlier message.
*Main changes include :*
>
>- Adding support for MATCH when creating new acl in a
I do not think we need to add the additional MATCH and ALL into the
Authorizer. Currently we have
- Literal - matches exactly
- Prefix - Will match patterns that are longer as long as the part
specified matches exactly.
The Trie proposal works well with just these two. The wildcard is
I have updated the KIP with results from the Trie implementation and they
are dramatic to say the least. For most searches they are at least an
order of magnitude faster and use less memory. The wildcard search is not
a regular expression but rather a file type wild card (*=1 or more
character, ?
Hi,
Just bumping this thread again. It seems no concerns have been raised so
far.
I'll request votes in 2 weeks.
Thanks,
Murali
On Tue, May 28, 2024 at 1:24 PM Muralidhar Basani <
muralidhar.bas...@aiven.io> wrote:
> Hi all,
>
> Any other suggestions or objections to the proposal?
>
> Thanks,
Hi all,
Any other suggestions or objections to the proposal?
Thanks,
Murali
On Thu, May 23, 2024 at 10:18 AM Muralidhar Basani <
muralidhar.bas...@aiven.io> wrote:
> Thanks Greg.
> I have updated KIP with details on optimisation of LITERAL too.
>
> Regarding limitations in performance by introd
Thanks Greg.
I have updated KIP with details on optimisation of LITERAL too.
Regarding limitations in performance by introducing MATCH is definitely a
question.
- By optimizing LITERAL and PREFIX we are gaining a few nano secs I think.
(described in kip)
- MATCH can be introduced only with a confi
Hi Murali,
I don't have a trie library in mind. I looked at the current
implementation of the StandardAuthorizer and found that we are already
benefiting from the prefix structure in the implementation [1]. The
current implementation appears to be a TreePSet [2].
Now, we've already made this trad
@greg which library of trie you were thinking of ? There is one in the
commons-collection I see.
On Fri, May 17, 2024 at 3:55 PM Claude Warren wrote:
> >
> > This has implications for execution complexity: If we can't compute
> > whether two patterns overlap, then we need to run both of them on
>
> This has implications for execution complexity: If we can't compute
> whether two patterns overlap, then we need to run both of them on each
> piece of input to test if they both match. Under the current
> LITERAL/PREFIX system, we can optimize execution with a trie, but that
> option wouldn't
Updated KIP with the provided suggestions from Greg and Claude.
We shall optimize matching acls with Trie based solutions for PREFIXED and
LITERAL acls. Basically we define trie structures, populate trie and
retrieve acls which seems very efficient.
This way, we aim to reduce the time to authorize
I think that a Trie solution for LITERAL and PREFIX would be faster than
attempting to use the wildcard search strategies on them. The reasons
being
- that the wildcard search strategy may return some non-matching (false
positive) patterns that have to be checked and ignored.
- the size
Thank you Haruki for the feedback.
There are certainly many such use cases where customers ended up creating
custom authorizers to handle these MATCH based patterns.
And regarding updating method matchingAcls, Claude updated the KIP
mentioning an approach, which definitely optimizes the flow, but c
I have updated KIP-1042 with a proposal for how to reduce the time spent
looking for matching wildcard patterns. Experimentally I see a reduction
of 66-75% execution time.
On Mon, May 6, 2024 at 9:03 PM Greg Harris
wrote:
> Hi Murali,
>
> Thanks for the KIP!
>
> I think I understand the motivat
Hi Murali,
Thanks for the KIP!
I think I understand the motivation for this KIP in situations where
there are a "cross product" of topics for two or more variables X and
Y, and want to write ACLs for each of the variable axes.
If you format your topics "X-Y-suffix", it's not easy to write rules
t
I have an idea for how to reduce the time for ACL lookups in general and
particularly where wildcards are involved using sequence
characterization techniques from bioinformatics. But I need a set of ACL
patterns and associated topics to test with.
On Fri, May 3, 2024 at 2:45 PM Haruki Okada wrot
Hi, Murali.
First, could you add the KIP-1042 to the index (
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals)
as well so that everyone can find it easily?
I took a look at the KIP, then I have 2 questions:
1. Though the new MATCH resource pattern type may reduce the
As I wrote in [1], the ACL evaluation algorithm needs to be specified with
respect to the specificity of the pattern so that we know exactly which if
*-accounts-* takes precedence over nl-accounts-* or visa versa.
I think that we should spell out that precedence is evaluated as follows:
1. Remove
Took me awhile to find it but the link to the KIP is
https://cwiki.apache.org/confluence/display/KAFKA/KIP-1042%3A+Support+for+wildcard+when+creating+new+acls
On Fri, May 3, 2024 at 10:13 AM Murali Basani
wrote:
> Hello,
>
> I'd like to propose a suggestion to our resource patterns in Kafka ACLs
Hello,
I'd like to propose a suggestion to our resource patterns in Kafka ACLs.
Currently, when adding new ACLs in Kafka, we have two types of resource
patterns for topics:
- LITERAL
- PREFIXED
However, when it comes to listing or removing ACLs, we have a couple more
options:
- MATCH
20 matches
Mail list logo
