Hi Ismael
Thank you for your help and quick response. You are right, the file IO
is blocking with Loom now. And small number threads might not having
benefits if they were just changed to virtual threads.
The virtual threads could help when you have a lot of tasks which are
executed in some thread pool. So each such task is executed in separate
virtual thread and could be yielded instead of blocking thread. Do you
think you have something similar in Kafka?
It would be great if you just point me to such tasks. So I could try to
execute them in virtual thread pool. If you don't think you have them or
they couldn't be easily "virtualized" it is fine. I understand that Loom
is still a separate project and it might be premature to think about any
significant changes related to it. So now I would like just to try to
use such changes to test Loom implementation itself.
Leonid
On 11/10/20 6:24 AM, Ismael Juma wrote:
Hi Leonid,
Thanks for looking into this. I think the main question is how
virtual threads would help here. Kafka tends to use a small number of
threads combined with non blocking IO, but there are some exceptions like
disk IO. My understanding is that the latter is still blocking even with
the latest builds of Loom. Is that right?
Ismael
On Mon, Nov 9, 2020 at 10:26 AM Leonid Mesnik <leonid.mes...@oracle.com>
wrote:
Hi
Currently, I am working on Loom project which enables virtual threads
for Java.
https://wiki.openjdk.java.net/display/loom/Main
As one of real-life tests it would be interesting to run Kafka using
virtual threads. However it might require some support in Kafka for
this. It is needed to add ability to start some threads as "virtual". Do
you know if anyone is interested and could help me with this?
Here are more details:
Basically, the virtual thread is a sub-class of java.lang.Thread. So it
is need to refactor code to avoid subclassing of thread and factorize
thread creation. I placed "example" fix what should be done to add
ability to run KafkaThread as virtual thread. It is just to demonstrate
the overall idea of changes.
https://urldefense.com/v3/__https://github.com/lmesnik/kafka/commit/872a2d5fd57b0c76878eece6c54c783897ccbf5e__;!!GqivPVa7Brio!MaLCXTeiTLzVpWU8BD3j4o6RxBqSOki307AHuhzHGrX1Q_9khpAC-9FTvrXyrcGUNA$
I want to check with you if it is a good approach for Kafka and are
there are other places to be updated. There is no plan to push such
support in mainline yet. Also, no plans to make any significant changes.
But if they want we could do it.
What do you think about this?
Leonid
