Igor, thanks for taking the time to look and to review the code. I regret
that I have not pushed the latest code, but I will do so and will see what
I can do about answering your Bloom filter related questions here.
How would an operator know or decide to change the configuration
> for the numb
Quick note: I renamed the example code. It is now at
https://github.com/Claudenw/kafka/blob/KIP-936/storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerIDQuotaManagerCache.java
On Thu, May 2, 2024 at 10:47 AM Claude Warren, Jr
wrote:
> Igor, thanks for taking the time
There is some question about whether or not we need the configuration
options. My take on them is as follows:
producer.id.quota.window.num No opinion. I don't know what this is used
for, but I suspect that there is a good reason to have it. It is not used
within the Bloom filter caching mechan
I think that if this is introduced (and perhaps even if it is not) we need
a clear ACL evaluation process.
I know we have both allow and deny, and that deny takes precedence over
allow.
But let's consider two scenarios
1. Unintended access.
Let's assume we start with the 6 topics Murali used in
As I wrote in [1], the ACL evaluation algorithm needs to be specified with
respect to the specificity of the pattern so that we know exactly which if
*-accounts-* takes precedence over nl-accounts-* or visa versa.
I think that we should spell out that precedence is evaluated as follows:
1. Remove
t easily?
> > >
> > > I took a look at the KIP, then I have 2 questions:
> > >
> > > 1. Though the new MATCH resource pattern type may reduce the effort of
> > > adding ACLs in some cases, do you have any concrete use case you are in
> > > mind? (Wh
First, a "mea culpa". In the KIP I state that several other KIPs are
rejected alternatives. That is not the case, they are addressing different
issues and would still work with KIP-1044. I will address this with an
update to the KIP.
Addressing the case where PID has not been seen before:
The
Igor,
Thanks for the well thought out comment. Do you have a suggestion for a
fast way to write to disk? Since the design requires random access perhaps
just a random access file?
Claude
On Thu, May 23, 2024 at 1:17 PM Igor Soarez wrote:
> Hi Claude,
>
> Thanks for writing this KIP. This iss
I give this a cautious +1 (non binding) as development may yield better
head wildcard results.
I think the adoption criteria for the ACL search needs to be specified in
the KIP. We do not have a good handle on how long the current searches
take. If the wildcard tests can be merged into a trie se
I think that if we put in a trie based system we should be able to halve
the normal searhc times and still be able to locate wild card matches very
quickly. Users should be warned that "head wildcard" matches are slow and
to use them sparingly. I am going to see if I can work out how to do
wildca
this code, I will complete the documentation
and fix the checkstyle and then open a pull request.
Claude
[1] https://github.com/Claudenw/kafka/pull/new/KIP-1042_Trie_Implementation
On Wed, Jul 3, 2024 at 2:21 PM Claude Warren, Jr
wrote:
> I think that if we put in a trie based system we should
; > > > LITERAL/PREFIX system, we can optimize execution with a trie, but
> >>> that
> >>> > > > option wouldn't be available to us with MATCH.
> >>> > > >
> >>> > > > The current system makes users evaluate a trade-off:
> >>> >
29, 2024 at 8:36 AM Claude Warren, Jr
wrote:
> I have updated the KIP with results from the Trie implementation and they
> are dramatic to say the least. For most searches they are at least an
> order of magnitude faster and use less memory. The wildcard search is not
> a regular expres
RAL and PREFIXED acls.
>
>
>- Define the Trie structure
>
>
>- Populate the Trie with ACLs
>
>
>- Retrieve ACLs using the Trie
>
> With this optimization, we hope to have a drastic reduced latency in the
> matchingAcls method, and it's much more efficient.
StandardAuthorizor.
-
https://github.com/Claudenw/kafka/compare/StandardAuthorizer_refactor...KIP-1042_trie_simplification
applies
the Trie changes to the authorizer refactor.
On Fri, Aug 2, 2024 at 10:07 AM Claude Warren, Jr
wrote:
> Proposed Changes
>> This KIP suggests to suppo
If there is an authorizer with no ACLs and
authorizeByResourceType(AuthorizableRequestContext
requestContext, AclOperation op, ResourceType resourceType) is called with
op = UNKNOWN or ANY, or resourceType = UKNOWN or ANY should an
IllegalArgumentException be thrown as it is when there are ACLs?
I
:DOH: Nevermind. Problem between keyboard and seat.
On Thu, Aug 15, 2024 at 8:36 AM Claude Warren, Jr
wrote:
> If there is an authorizer with no ACLs and
> authorizeByResourceType(AuthorizableRequestContext
> requestContext, AclOperation op, ResourceType resourceType) is called
Greetings,
I have been working on Apache RAT recently and I noticed that Kafka has a
very nice XSLT to convert the Rat output to an HTML document.
I know there is not a legal or licensing issue but I am asking if there are
any objections to my taking the .gradle/resources/rat-output-to-html.xsl
f
Should a ResourcePatternFilter that has a PatternType of ANY and a name of
WILDCARD_RESOURCE not match any Acls? I think this is a bug.I am writing
a series of tests to ensure that I have implemented everything correctly in
the Trie implementation and this has come up.
public boolean matches(Res
I am new to the Kafka codebase so please excuse any ignorance on my part.
When a dead letter queue is established is there a process to ensure that
it at least is defined with the same ACL as the original queue? Without
such a guarantee at the start it seems that managing dead letter queues
will
URL: https://issues.apache.org/jira/browse/KAFKA-17423
The above is an improvement to Kafka to replace the sorted list ACL
implementation with a Trie based implementation. I have an implementation
that passes all the tests, including the new ones in
KAFKA-17316 (pull request https://github.com/ap
aude
On Fri, Aug 23, 2024 at 9:43 PM Colin McCabe wrote:
> On Sat, Jul 27, 2024, at 04:20, Claude Warren, Jr wrote:
> > I have updated the KIP with results from the Trie implementation and they
> > are dramatic to say the least. For most searches they are at least an
> > order
y have had,
Claude
On Thu, Aug 29, 2024 at 6:51 PM Colin McCabe wrote:
> On Thu, Aug 29, 2024, at 01:34, Claude Warren, Jr wrote:
>
> Colin,
>
> Thanks for your insightful comments. I came to the same conclusion.
>
> I do have 2 Jira tickets to simplify some of t
r these things. Does that make sense? I don't
> see another path to doing it compatibly. I certainly wouldn't want to
> create a "Python 2 vs. Python 3" type situation where people get stuck on
> an older authorizer fork because the new one requires globs and they can'
I have been working on implementing a Trie structure to store ACLs and
improve the performance in the metadata/authorization code. The upshot of
this was that I found it very difficult to determine if the implementation
was correctly reimplementing the current implementation.
My goal was to simpl
After some discussion about the earlier KIP-1042 we have rewritten it to
focus on the implementation of a GLOB pattern type. Please review and
comment.
We have removed all discussion of the Trie implementation and focus on what
is required for the GLOB implementation. The KIP does assume that th
a pull request to satisfy KAFKA-17423 that contains only
the new implementation.
Claude
On Mon, Sep 2, 2024 at 9:09 AM Claude Warren, Jr
wrote:
> I have been working on implementing a Trie structure to store ACLs and
> improve the performance in the metadata/authorization code. The ups
I am working on a replacement for the StandardAuthorizer and my
implementation DENIED while the standard implementation ALLOWED. In
reading the specs I thought it should be DENIED. But your statement makes
it clear that I misread.
Thank you,
Claude
On Tue, Sep 3, 2024 at 1:14 PM Rajini Sivaram
Followup: If ALLOW_EVERYONE_IF_NO_ACL_IS_FOUND_CONFIG = "true" then
authorizeByResourceType should return true in all cases since the user
would have access for any operation on any undefined topic?
On Tue, Sep 3, 2024 at 2:08 PM Claude Warren, Jr
wrote:
> I am working on a re
Followup2: your answer speaks directly to "WRITE" access. My example was
READ access. So the question method is answering then is: Does the user
have access to READ any TOPIC? And that is further restricted by the
requestContext host is it not?
On Tue, Sep 3, 2024 at 2:10 PM Claude
r
> duplicating this file to be within the RAT project. The place to ask is
> probably in the RAT project itself -- I don't know if this is something
> they'd like to include or not (hopefully yes.)
>
> cheers,
> Colin
>
>
> On Thu, Aug 15, 2024, at 04:54, Claud
I like this idea. I'm not sure what the section should be called but It
should spell out what changes from a customer (I don't like the term user,
drug dealers have users -- we should have customers) point of view and from
a developer point of view.
I can see cases where the change is not visible
Greetings,
Please excuse any ignorance but my reading of the code indicates that
SinkConnectorConfig should be used for the configuration of sink connectors.
However, there is no mechanism to add additional items to the configuration
as the ConfigDef is created during construction and passed as a
I have been working on a similar process for Apache RAT and Commons CLI
(help module). The help option in RAT 0.17-SNAPSHOT generates the text for
the help options partly from the new commons-cli help module and partly
from Enums and similar in the code.
For the RAT documentation we produce fragm
34 matches
Mail list logo
