Hi Hanyu,
Thanks for the KIP!
+1 (non-binding)
Hao
On Thu, Nov 2, 2023 at 1:29 PM Bill Bejeck wrote:
> Hi Hanyu,
>
> Thanks for the KIP this LGTM.
> +1 (binding)
>
> Thanks,
> Bill
>
>
>
> On Wed, Nov 1, 2023 at 1:07 PM Hanyu (Peter) Zheng
> wrote:
>
> > Hello everyone,
> >
> > I would like t
e, it will throw a
> > exception,
> > the exception like this:
> > java.lang.IllegalArgumentException: Cannot get result for failed query.
> >
> > Sincerely,
> > Hanyu
> >
> > On Thu, Oct 26, 2023 at 3:45 PM Hao Li wrote:
> >
> >> Thanks fo
Thanks for the KIP Hanyu! One question: why not return an iterator of
`ValueAndTimestamp` for `TimestampedKeyQuery`? I suppose for a
ts-kv-store, there could be multiple timestamps associated with the same
key?
Hao
On Thu, Oct 26, 2023 at 10:23 AM Matthias J. Sax wrote:
> Would we really get a
Seems late. But +1 (non-binding).
Thanks for the KIP!
Hao
On Tue, Oct 17, 2023 at 10:27 AM Hanyu (Peter) Zheng
wrote:
> Hi all,
>
> This voting thread has been open for over 72 hours and has received enough
> votes. Therefore, the vote will be closed now.
>
> +3 binding votes
>
> KIP-985 has P
Hi Hanyu,
Thanks for the KIP! Seems there are already a lot of good discussions. I
only have two comments:
1. Please make it clear in
```
/**
* Interactive range query using a lower and upper bound to filter the
keys returned.
* @param lower The key that specifies the lower bound of
[
https://issues.apache.org/jira/browse/KAFKA-15054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li resolved KAFKA-15054.
Fix Version/s: 3.6.0
Resolution: Fixed
> Add configs and logic to decide if rack aware assignm
[
https://issues.apache.org/jira/browse/KAFKA-15025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li resolved KAFKA-15025.
Fix Version/s: 3.6.0
Resolution: Fixed
> Implement min-cost flow without balancing tasks for s
[
https://issues.apache.org/jira/browse/KAFKA-15027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li resolved KAFKA-15027.
Fix Version/s: 3.6.0
Resolution: Fixed
> Implement rack aware assignment for standby ta
[
https://issues.apache.org/jira/browse/KAFKA-15024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li resolved KAFKA-15024.
Fix Version/s: 3.6.0
Resolution: Fixed
> Add cost function for task/cli
[
https://issues.apache.org/jira/browse/KAFKA-15023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li resolved KAFKA-15023.
Fix Version/s: 3.6.0
Resolution: Fixed
> Get rack information for source topic partitions fo
Hi Satish,
KIP-925 was accepted and currently under implementation. I just added it to
the release plan.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assignment+in+Kafka+Streams
Thanks,
Hao
On Thu, Jul 20, 2023 at 6:18 AM Christo Lolov
wrote:
> Hello!
>
> A cou
t; > > +1 (non-binding)
> > > >
> > > > Thank you Hao!
> > > >
> > > > Colt McNealy
> > > >
> > > > *Founder, LittleHorse.dev*
> > > >
> > > >
> > > > On Tue, May 30, 2023 at 9:50 AM Hao Li
>
ou should be able to say "retest this please" in a PR comment
> to trigger it. Apparently, that doesn't work anymore, though. I'll file an
> Infra ticket for it.
>
> Thanks all,
> John
>
> On Fri, Jun 2, 2023, at 18:46, Hao Li wrote:
> > Hi Luke,
>
Hi Luke,
Sorry for the late reply. Can you also add me to the whitelist? I believe
I'm supposed to be there as well. Matthias and John can vouch for me :) My
github ID is "lihaosky".
Thanks,
Hao
On Fri, Jun 2, 2023 at 4:43 PM Greg Harris
wrote:
> Luke,
>
> I see that the PR has been merged, bu
Hao Li created KAFKA-15054:
--
Summary: Add configs and logic to decide if rack aware assignment
should be enabled
Key: KAFKA-15054
URL: https://issues.apache.org/jira/browse/KAFKA-15054
Project: Kafka
ave observed that changing partition leadership can trigger
> ProducerFencedExceptions in EOS, which causes a state restoration.
>
> Colt McNealy
>
> *Founder, LittleHorse.dev*
>
>
> On Thu, Jun 1, 2023 at 10:02 AM Hao Li wrote:
>
> > Hi Bruno,
> >
> > d
you think about dropping config rack.aware.assignment.enabled
> and add value NONE to the enum for the possible values of config
> rack.aware.assignment.strategy?
>
> Best,
> Bruno
>
> On 31.05.23 23:31, Hao Li wrote:
> > Hi all,
> >
> > I've updated the KIP b
dependently.
>
> If users are able to plug in the StickyAssignor if they experience
> problems, this is good enough for me w.r.t my
> other concern regarding task shuffling
>
> On Wed, May 31, 2023 at 12:09 PM Hao Li wrote:
>
> > Hi all,
> >
> > Thanks for the
Hi all,
I've updated the KIP based on the feedback. Major changes I made:
1. Add rack aware assignment to `StickyTaskAssignor`
2. Reject `Prefer reliability and then find optimal cost` option in standby
task assignment.
On Wed, May 31, 2023 at 12:09 PM Hao Li wrote:
> Hi all,
>
&
re available.
>
> Building off of the above, the HAAssignor hasn't worked out perfectly for
> everybody up until now,
> given that we are only adding complexity to it now, on the flipside I would
> hesitate to try and force
> everyone to use it if they want to upgrade. We added
Hi all,
I'd like to open the vote for KIP-925: rack aware task assignment in Kafka
Streams. The link for the KIP is
https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assignment+in+Kafka+Streams
.
--
Thanks,
Hao
Hao Li created KAFKA-15027:
--
Summary: Implement rack aware assignment for standby tasks
Key: KAFKA-15027
URL: https://issues.apache.org/jira/browse/KAFKA-15027
Project: Kafka
Issue Type: Sub-task
Hao Li created KAFKA-15025:
--
Summary: Implement min-cost flow without balancing tasks for same
subtopology
Key: KAFKA-15025
URL: https://issues.apache.org/jira/browse/KAFKA-15025
Project: Kafka
Hao Li created KAFKA-15026:
--
Summary: Implement min-cost flow balancing tasks for same
subtopology
Key: KAFKA-15026
URL: https://issues.apache.org/jira/browse/KAFKA-15026
Project: Kafka
Issue Type
Hao Li created KAFKA-15024:
--
Summary: Add cost function for task/client
Key: KAFKA-15024
URL: https://issues.apache.org/jira/browse/KAFKA-15024
Project: Kafka
Issue Type: Sub-task
Hao Li created KAFKA-15023:
--
Summary: Get rack information for source topic partitions for a
task
Key: KAFKA-15023
URL: https://issues.apache.org/jira/browse/KAFKA-15023
Project: Kafka
Issue Type
Hao Li created KAFKA-15022:
--
Summary: Support rack aware task assignment in Kafka streams
Key: KAFKA-15022
URL: https://issues.apache.org/jira/browse/KAFKA-15022
Project: Kafka
Issue Type
ut I'd feel confident about it if
> we had a little more detail regarding how we are going to verify that the
> new optimizer is actually going to produce more optimal plans than the
> existing assigner we have today.
>
> Thanks again!
> -John
>
> On 2023/05/22 16:49:22 Ha
pe of this KIP, or is it worth introducing a
> kafka-streams-generate-rebalance-proposal.sh tool?
>
> Colt McNealy
> *Founder, LittleHorse.io*
>
>
> On Tue, May 9, 2023 at 4:03 PM Hao Li wrote:
>
> > Hi all,
> >
> > I have submitted KIP-925 to add r
Hao Li created KAFKA-14981:
--
Summary: Set `group.instance.id` in streams consumer so that
rebalance will not happen if a instance is restarted
Key: KAFKA-14981
URL: https://issues.apache.org/jira/browse/KAFKA-14981
Hi all,
I have submitted KIP-925 to add rack awareness logic in task assignment in
Kafka Streams and would like to start a discussion:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-925%3A+Rack+aware+task+assignment+in+Kafka+Streams
--
Thanks,
Hao
[
https://issues.apache.org/jira/browse/KAFKA-14395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li resolved KAFKA-14395.
Resolution: Done
> Add config to configure client supplier for KafkaStre
Hi all,
I updated the KIP to add a `getKafkaClientSupplier` method in
`StreamsConfig`. Let me know if you have any concerns.
Thanks,
Hao
On Wed, Nov 30, 2022 at 10:26 AM Hao Li wrote:
> Hi all,
>
> Thanks for the vote. The vote passed with 4 binding votes (John, Matthias,
> Soph
:39 AM, John Roesler wrote:
> >>> I'm +1 (binding)
> >>>
> >>> Thanks for the KIP!
> >>> -John
> >>>
> >>> On 2022/11/17 21:06:29 Hao Li wrote:
> >>>> Hi all,
> >>>>
> >>>> I wo
Yes. I've updated the KIP with suggested changes.
Thanks,
Hao
On Thu, Nov 17, 2022 at 7:56 PM Matthias J. Sax wrote:
> SG.
>
> Can we clarify/document the behavior on the KIP?
>
>
> -Matthias
>
> On 11/15/22 4:19 PM, Hao Li wrote:
> > Thanks for the questi
Hi all,
I would like start a vote on KIP-884:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-884%3A+Add+config+to+configure+KafkaClientSupplier+in+Kafka+Streams
Thanks,
Hao
[
https://issues.apache.org/jira/browse/KAFKA-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li resolved KAFKA-13785.
Resolution: Done
> Support emit final result for windowed aggregat
Hao Li created KAFKA-14395:
--
Summary: Add config to configure client supplier for KafkaStreams
Key: KAFKA-14395
URL: https://issues.apache.org/jira/browse/KAFKA-14395
Project: Kafka
Issue Type
If we do this, should we change the config name to
> `default.client.supplier` and not make it `null`, but set the default
> supplier we use currently?
>
> This way, the config and code would behave the same as other configs
> like `default.timestamp.extractor` and similar.
>
>
Hi all,
I have submitted KIP-884 to add config to configure KafkaClientSupplier and
would like to start a discussion:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-884%3A+Add+config+to+configure+KafkaClientSupplier+in+Kafka+Streams
--
Thanks,
Hao
Hao Li created KAFKA-14117:
--
Summary: Flaky Test
DynamicBrokerReconfigurationTest.testKeyStoreAlter
Key: KAFKA-14117
URL: https://issues.apache.org/jira/browse/KAFKA-14117
Project: Kafka
Issue
[
https://issues.apache.org/jira/browse/KAFKA-12566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li reopened KAFKA-12566:
Somehow this is still happening:
```
[2022-07-27T11:32:04.461Z] MirrorConnectorsIntegrationSSLTest
it to the KIP.
Thanks,
Hao
On Thu, Mar 24, 2022 at 4:17 PM Hao Li wrote:
> The vote passed with 5 binding votes from John, Guozhang, Bruno, Matthias
> and Bill.
>
> Thanks all for the feedback and vote.
>
> Hao
>
> On Thu, Mar 24, 2022 at 2:20 PM Bill Bejeck wrote:
>
in the metrics? Is that
> measured as the time difference between "when a final result record COULD
> be emitted" and "when that final result record is actually emitted"?
> Anyways, maybe it's better to elaborate clearly on how the latency is
> measured.
>
>
Hao Li created KAFKA-13824:
--
Summary: Pass time object from constructor so that we can mock it
if needed
Key: KAFKA-13824
URL: https://issues.apache.org/jira/browse/KAFKA-13824
Project: Kafka
> behavior of emit-final windowed aggregations, then we should
> > make sure that the metric name says as much. Maybe:
> >
> > emit-final-records-[rate|total]
> > emit-final-latency-[avg|max]
> >
> > Thanks!
> > -John
> >
> > On Mon, Apr 1
25%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
Can you take a look and see if you have any concerns for the metrics?
Thanks,
Hao
On Fri, Mar 25, 2022 at 8:27 AM Hao Li wrote:
> Got it. Forgot that. Yeah, it’s still open and ppl can still vote. Thanks
> for reminding!
Hao Li created KAFKA-13817:
--
Summary: Schedule nextTimeToEmit to system time every time instead
of just once
Key: KAFKA-13817
URL: https://issues.apache.org/jira/browse/KAFKA-13817
Project: Kafka
Hao Li created KAFKA-13800:
--
Summary: Remove force cast of TimeWindowKStreamImpl in tests of
https://github.com/apache/kafka/pull/11896
Key: KAFKA-13800
URL: https://issues.apache.org/jira/browse/KAFKA-13800
Hao Li created KAFKA-13785:
--
Summary: Support emit final result for windowed aggregation
Key: KAFKA-13785
URL: https://issues.apache.org/jira/browse/KAFKA-13785
Project: Kafka
Issue Type
Got it. Forgot that. Yeah, it’s still open and ppl can still vote. Thanks for
reminding!
Hao Li
> On Mar 25, 2022, at 8:22 AM, Guozhang Wang wrote:
>
> Hello Hao,
>
> According to bylaws the voting has to last for at least 72 business hours.
> So let's wait a bit lon
atthias J. Sax wrote:
>
> > +1 (binding)
> >
> >
> > On 3/24/22 1:52 PM, Hao Li wrote:
> > > I hit reply on my phone in the mail app and changed the title and text
> > > hoping it will start a new thread. Apparently it doesn't work...
> > >
>
r 24, 2022 at 5:13 PM Matthias J. Sax wrote:
>
> > +1 (binding)
> >
> >
> > On 3/24/22 1:52 PM, Hao Li wrote:
> > > I hit reply on my phone in the mail app and changed the title and text
> > > hoping it will start a new thread. Apparently it doesn&#x
bit weird how it shows e-mails in threads.
>
> Anyways, thank you for the KIP and your patience!
>
> +1 (binding)
>
> Best,
> Bruno
>
> On 24.03.22 17:36, Hao Li wrote:
> > Thanks John and Guozhang. Didn't realize I used this discussion thread
> for
> >
Hi all,
I'd like to start a vote on Kafka Streams KIP-825:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
Thanks,
Hao
t 9:20 AM John Roesler wrote:
>
> > Thanks, Hao!
> >
> > I'm +1 (binding)
> >
> > -John
> >
> > On Wed, 2022-03-23 at 22:25 -0700, Hao Li wrote:
> > > Hi all,
> > >
> > > I'd like to start a vote on Kafka Streams K
Hi all,
I'd like to start a vote on Kafka Streams KIP-825:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
any
> > options, Hao could make the final call about the namings.
> >
> >
> > Guozhang
> >
> > On Wed, Mar 23, 2022 at 1:49 PM Hao Li wrote:
> >
> >> For
> >>
> >> stream
> >>.groupBy(..)
> >>.w
ll prefers option 1, then (as long
> > as we don't think it's a horrible mistake), I think we
> > should just let him go for it.
> >
> > Speaking of which, after reviewing the responses regarding
> > deprecating `Suppressed#onWindowClose`, I still think we
>
t; > Thanks for the great summary Hao. I'm still learning towards option 2)
> > here, and I'm in favor of `trigger` as function name, and `Triggered` as
> > config class name (mainly to be aligned with other config class names).
> > Also want to see other'
;
> > >> One thing I noticed with option 1 is that all other methods in the
> > >> example above are operations on data. `groupBy()` groups,
> `windowedBy()`
> > >> partitions, `aggregate()` computes the aggregate, `mapValues()` maps
> > >> values, even `sup
er()` do? `trigger()` seems a config lost among operations.
> >>
> >> However, if we do not want to restrict ourselves to only use methods
> >> when we want to specify operations on data, I have the following
> proposal:
> >>
> >> stream
> >> .group
> Hi Hao,
> > >> >
> > >> > For 2), I think it's a good idea in general to use a separate
> > function on
> > >> > the Time/SessionWindowedKStream itself, to achieve the same effect
> > that,
> > >> > for now, the emitti
fig either.
> >
> > Option 1: Your proposal is?
> >
> >stream
> > .groupByKey()
> > .windowBy(TimeWindow.ofSizeNoGrace(...))
> > .configure(EmitConfig.emitFinal()
> > .count()
> >
> > Does not change my argument that it seems
.groupByKey()
> .windowBy(
>TimeWindow.ofSizeNoGrace(...)
> )
> .count(EmitConfig.emitFinal())
>
> The API surface area that need to be updated might be larger for this
> case though...
>
>
> -Matthias
>
>
>
> On 3/14/22 9:21 PM,
for this KIP maybe we do not have to add them but can delay for later after
> we've learned the best way to layout. LMK what do you think?
>
>
>
> Guozhang
>
>
>
> On Fri, Mar 11, 2022 at 2:13 PM Hao Li wrote:
>
> > Hi Dev team,
> >
> > I'd lik
Hi Dev team,
I'd like to start a discussion thread on Kafka Streams KIP-825:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-825%3A+introduce+a+new+API+to+control+when+aggregated+results+are+produced
This KIP is aimed to add new APIs to support outputting final aggregated
results for window
[
https://issues.apache.org/jira/browse/KAFKA-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Li resolved KAFKA-13164.
Resolution: Cannot Reproduce
Closing this as I can't reproduce the issue
> State store is att
68 matches
Mail list logo
