One thing I can think of is the ACL operations (create / delete ACLs). They are much more less frequently used other than the produce/consume/topic creation so I’m not sure if it’s valuable to use as a benchmark.
- Cheng > On Nov 11, 2020, at 4:33 PM, Leonid Mesnik <leonid.mes...@oracle.com> wrote: > > Hi Ismael > > Thank you for your help and quick response. You are right, the file IO is > blocking with Loom now. And small number threads might not having benefits > if they were just changed to virtual threads. > > The virtual threads could help when you have a lot of tasks which are > executed in some thread pool. So each such task is executed in separate > virtual thread and could be yielded instead of blocking thread. Do you think > you have something similar in Kafka? > > It would be great if you just point me to such tasks. So I could try to > execute them in virtual thread pool. If you don't think you have them or they > couldn't be easily "virtualized" it is fine. I understand that Loom is still > a separate project and it might be premature to think about any significant > changes related to it. So now I would like just to try to use such changes to > test Loom implementation itself. > > Leonid > > On 11/10/20 6:24 AM, Ismael Juma wrote: >> Hi Leonid, >> >> Thanks for looking into this. I think the main question is how >> virtual threads would help here. Kafka tends to use a small number of >> threads combined with non blocking IO, but there are some exceptions like >> disk IO. My understanding is that the latter is still blocking even with >> the latest builds of Loom. Is that right? >> >> Ismael >> >> On Mon, Nov 9, 2020 at 10:26 AM Leonid Mesnik <leonid.mes...@oracle.com> >> wrote: >> >>> Hi >>> >>> >>> Currently, I am working on Loom project which enables virtual threads >>> for Java. >>> >>> https://wiki.openjdk.java.net/display/loom/Main >>> >>> As one of real-life tests it would be interesting to run Kafka using >>> virtual threads. However it might require some support in Kafka for >>> this. It is needed to add ability to start some threads as "virtual". Do >>> you know if anyone is interested and could help me with this? >>> >>> Here are more details: >>> >>> Basically, the virtual thread is a sub-class of java.lang.Thread. So it >>> is need to refactor code to avoid subclassing of thread and factorize >>> thread creation. I placed "example" fix what should be done to add >>> ability to run KafkaThread as virtual thread. It is just to demonstrate >>> the overall idea of changes. >>> >>> >>> https://urldefense.com/v3/__https://github.com/lmesnik/kafka/commit/872a2d5fd57b0c76878eece6c54c783897ccbf5e__;!!GqivPVa7Brio!MaLCXTeiTLzVpWU8BD3j4o6RxBqSOki307AHuhzHGrX1Q_9khpAC-9FTvrXyrcGUNA$ >>> >>> <https://urldefense.com/v3/__https://github.com/lmesnik/kafka/commit/872a2d5fd57b0c76878eece6c54c783897ccbf5e__;!!GqivPVa7Brio!MaLCXTeiTLzVpWU8BD3j4o6RxBqSOki307AHuhzHGrX1Q_9khpAC-9FTvrXyrcGUNA$> >>> >>> >>> I want to check with you if it is a good approach for Kafka and are >>> there are other places to be updated. There is no plan to push such >>> support in mainline yet. Also, no plans to make any significant changes. >>> But if they want we could do it. >>> >>> What do you think about this? >>> >>> Leonid
