k-apol commented on code in PR #19913:
URL: https://github.com/apache/kafka/pull/19913#discussion_r2147033788
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java:
##########
@@ -461,15 +469,22 @@ public Set<String> makeReady(final Map<String,
InternalTopicConfig> topics) {
// have existed with the expected number of partitions, or some create
topic returns fatal errors.
log.debug("Starting to validate internal topics {} in partition
assignor.", topics);
- long currentWallClockMs = time.milliseconds();
+ final long currentWallClockMs = time.milliseconds();
final long deadlineMs = currentWallClockMs + retryTimeoutMs;
+ final long initDeadlineMs = currentWallClockMs +
this.initTimeout.toMillis();
Set<String> topicsNotReady = new HashSet<>(topics.keySet());
final Set<String> newlyCreatedTopics = new HashSet<>();
while (!topicsNotReady.isEmpty()) {
final Set<String> tempUnknownTopics = new HashSet<>();
topicsNotReady = validateTopics(topicsNotReady, topics,
tempUnknownTopics);
+
+ if (this.isManualInternalTopicConfig && !this.isInitializing) {
+ throw new MissingInternalTopicsException("Internal topic
configuration set to MANUAL. \n" +
+ "You must call init() to setup internal topics.", new
ArrayList<>(topicsNotReady));
+ }
+
newlyCreatedTopics.addAll(topicsNotReady);
if (!topicsNotReady.isEmpty()) {
Review Comment:
I think it's a good idea, we are making this method work really hard. He
could use some pals to help him out
##########
streams/src/main/java/org/apache/kafka/streams/processor/internals/InternalTopicManager.java:
##########
@@ -461,15 +469,22 @@ public Set<String> makeReady(final Map<String,
InternalTopicConfig> topics) {
// have existed with the expected number of partitions, or some create
topic returns fatal errors.
log.debug("Starting to validate internal topics {} in partition
assignor.", topics);
- long currentWallClockMs = time.milliseconds();
+ final long currentWallClockMs = time.milliseconds();
final long deadlineMs = currentWallClockMs + retryTimeoutMs;
+ final long initDeadlineMs = currentWallClockMs +
this.initTimeout.toMillis();
Set<String> topicsNotReady = new HashSet<>(topics.keySet());
final Set<String> newlyCreatedTopics = new HashSet<>();
while (!topicsNotReady.isEmpty()) {
final Set<String> tempUnknownTopics = new HashSet<>();
topicsNotReady = validateTopics(topicsNotReady, topics,
tempUnknownTopics);
+
+ if (this.isManualInternalTopicConfig && !this.isInitializing) {
Review Comment:
> Is this check actually correct here? `validateTopics()` above would return
all topics that we need to create, so if `topicsNotReady` is empty here, all
topic we did expect exist and we should not throw an exception. Should it be(?):
>
> ```
> if (!topicsNotReady.isEmpty() && isManualInternalTopicConfig) {
> throw new...
> }
> ```
>
> (for this case, we would also move the check further below inside the
already existing block that tried to creates internal topics L490):
>
> ```
> if (!topicsNotReady.isEmpty()) {
> if (isManualInternalTopicConfig) {
> throw new ...
> }
>
> final Set<NewTopic> newTopics = new HashSet<>();
> ```
My thinking was this,
if `(!topicsNotReady.isEmpty())` evaluates to true (the break condition for
the while loop), we are saying **there are some topics to create**.. but, if I
understand your point, that decision is not actually determined until validate
finishes executing?
Validate could return a result that says '**there's nothing to create', **
but the way this check stands, it would fail prematurely. If we decide to fail,
we would need to be certain that there is a need to actually create topics to
reach the desired state. I think capturing this decision in a variable could
make this more readable in a future refactor.
I will read over this and think on it, I think you may be right
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
