Talked to Ivan in private, and came up with the following per-language summary:
* Java: sync and async *. NET: only async * Python: sync and async * JavaScript: only async (sync is not possible) Thin clients in other languages are to be discussed. On Thu, Sep 9, 2021 at 11:49 AM Ivan Daschinsky <ivanda...@gmail.com> wrote: > >> You can mix them easily. > This is far from easily (you have already mentioned continuation problem), > but for i.e. in python it is absolutely not. > For kotlin it is a little bit easier, but also not fluent and a little bit > ugly. > > чт, 9 сент. 2021 г. в 11:37, Pavel Tupitsyn <ptupit...@apache.org>: > > > Ivan, > > > > > Pavel, is it really true, that in .NET sync versions of libraries and > > tools > > > are completely eliminated? > > > > Far from being eliminated, but there is a movement in this direction. > > For example, HttpClient [1] only provides async variants for most of the > > methods. > > > > Exposing sync wrappers for async methods is not recommended [2]. > > There is a good explanation of potential threading issues there, and it > can > > be applied to Java too. > > > > > > > you cannot mix both functions easily > > > > You can mix them easily. > > C#: table.GetAsync(k).Result or > table.GetAsync(k).GetAwaiter().GetResult() > > (different exception handling) > > Java: table.getAsync(k).get() > > > > The same thing we do in sync wrapper for async method is now in the user > > code. > > Which is better for two reasons: > > - users won't accidentally use sync API when async API should be used > > - when they really have to use sync API, potentially dangerous behavior > is > > not hidden > > > > > > [1] > > > > > https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-5.0#methods > > [2] > > > > > https://devblogs.microsoft.com/pfxteam/should-i-expose-synchronous-wrappers-for-asynchronous-methods/ > > > > On Thu, Sep 9, 2021 at 11:16 AM Ivan Daschinsky <ivanda...@gmail.com> > > wrote: > > > > > >> 2. In languages with proper async support (async-await, etc.), we > can > > > skip sync API altogether. > > > It sounds pretty strange, as for me. Usually you cannot mix both > > functions > > > easily, it is called blue-red functions problem [1] > > > In python you definitely cannot do sync over async, for example > > > (principally can, but nobody do that because of mediocre performance of > > > that solution). And many developers > > > still prefer writing code withou asyncio at all. > > > > > > Pavel, is it really true, that in .NET sync versions of libraries and > > tools > > > are completely eliminated? > > > > > > [1] -- > > > > https://elizarov.medium.com/how-do-you-color-your-functions-a6bb423d936d > > > > > > ср, 8 сент. 2021 г. в 22:33, Pavel Tupitsyn <ptupit...@apache.org>: > > > > > > > To put it another way: > > > > - true sync operation completes by itself > > > > - sync-over-async operation requires another thread to complete > > > > > > > > On Wed, Sep 8, 2021 at 10:15 PM Pavel Tupitsyn <ptupit...@apache.org > > > > > > wrote: > > > > > > > > > Val, > > > > > > > > > > That's exactly what I have in mind. > > > > > Yes, we block the user thread, but then we should unblock it by > > > > completing > > > > > the future. > > > > > We can't complete the future from a Netty thread [1], so we'll need > > > some > > > > > other thread from some executor. > > > > > If there are no threads available (because they are blocked by the > > sync > > > > > API above), the future won't complete => deadlock. > > > > > > > > > > [1] > > > > > > > > > > > > > > > https://lists.apache.org/thread.html/r528659381d983a177d779f56ef3d7da6fe17eb3504383f5f87727514%40%3Cdev.ignite.apache.org%3E > > > > > > > > > > On Wed, Sep 8, 2021 at 9:40 PM Valentin Kulichenko < > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > >> Pavel, > > > > >> > > > > >> I might be missing something - could you please elaborate a little > > > more? > > > > >> > > > > >> When I say "sync on top of async", I basically mean that (for > > example) > > > > >> 'insert(..)' is equivalent to 'insertAsync(..).join()'. In my > > > > >> understanding, it only blocks the user's thread. > > > > >> > > > > >> Am I wrong? Or you have a different implementation in mind? > > > > >> > > > > >> -Val > > > > >> > > > > >> On Wed, Sep 8, 2021 at 12:50 AM Pavel Tupitsyn < > > ptupit...@apache.org> > > > > >> wrote: > > > > >> > > > > >> > Val, > > > > >> > > > > > >> > Agree with your points. > > > > >> > > > > > >> > > > > > >> > > async API should be primary > > > > >> > > > > > >> > It should be noted that all our APIs are inherently async, > > > > >> > because thin client is implemented asynchronously. > > > > >> > > > > > >> > > > > > >> > > with the sync version build on top > > > > >> > > > > > >> > We should document somehow that sync APIs are based on async > ones, > > > > >> > because this may be dangerous in some use cases. > > > > >> > > > > > >> > For example, as a user, I may have a thread pool of 4 threads > for > > > > >> > Ignite-related usage, that is also set as > > asyncContinuationExecutor > > > > [1]. > > > > >> > Now if I run a lot of concurrent Ignite requests using sync API, > > > all 4 > > > > >> > threads will end up blocked on CompletableFutures. > > > > >> > When one of the operations completes, we enqueue the completion > to > > > > that > > > > >> > same thread pool, but all threads are blocked on sync APIs, > > > resulting > > > > >> in a > > > > >> > deadlock. > > > > >> > > > > > >> > This can be prevented by using a different > > > asyncContinuationExecutor, > > > > >> but > > > > >> > sync API users won't be usually aware of this. > > > > >> > > > > > >> > > > > > >> > [1] https://issues.apache.org/jira/browse/IGNITE-15359 > > > > >> > > > > > >> > On Wed, Sep 8, 2021 at 10:04 AM Courtney Robinson < > > > > >> > courtney.robin...@hypi.io> > > > > >> > wrote: > > > > >> > > > > > >> > > Hi Val, > > > > >> > > > > > > >> > > I'd highly support an async first API based on CompletionStage > > > > >> > > < > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/CompletionStage.html > > > > >> > > > > > > > >> > > or > > > > >> > > its subtypes like CompletableFuture. > > > > >> > > In Ignite 2 we've written a wrapper library around > IgniteFuture > > to > > > > >> > provide > > > > >> > > CompletionStage instead because many of the newer libs we use > > > > support > > > > >> > this. > > > > >> > > If Ignite 3 went this way it'd remove a lot of boiler > > > plate/wrapper > > > > >> that > > > > >> > we > > > > >> > > wrote to get what you're suggesting here. > > > > >> > > > > > > >> > > Regards, > > > > >> > > Courtney Robinson > > > > >> > > Founder and CEO, Hypi > > > > >> > > Tel: ++44 208 123 2413 (GMT+0) <https://hypi.io> > > > > >> > > > > > > >> > > <https://hypi.io> > > > > >> > > https://hypi.io > > > > >> > > > > > > >> > > > > > > >> > > On Wed, Sep 8, 2021 at 12:44 AM Valentin Kulichenko < > > > > >> > > valentin.kuliche...@gmail.com> wrote: > > > > >> > > > > > > >> > > > Igniters, > > > > >> > > > > > > > >> > > > I would like to gather some opinions on whether we want to > > focus > > > > on > > > > >> > sync > > > > >> > > vs > > > > >> > > > async APIs in Ignite 3. > > > > >> > > > > > > > >> > > > Here are some initial considerations that I have: > > > > >> > > > 1. Ignite 2.x is essentially "sync first". Async APIs exist, > > but > > > > >> they > > > > >> > use > > > > >> > > > non-standard IgniteFuture and provide counterintuitive > > > guarantees. > > > > >> In > > > > >> > my > > > > >> > > > experience, they significantly lack usability, and because > of > > > that > > > > >> are > > > > >> > > > rarely used. > > > > >> > > > 2. In general, however, async execution becomes more and > more > > > > >> > prominent. > > > > >> > > > Something we can't ignore if we want to create a modern > > > framework. > > > > >> > > > 3. Still, async support in Java is very limited (especially > if > > > > >> compared > > > > >> > > to > > > > >> > > > other languages, like C# for example). > > > > >> > > > > > > > >> > > > My current position is the following (happy to discuss): > > > > >> > > > 1. We should pay more attention to async APIs. As a general > > > rule, > > > > >> async > > > > >> > > API > > > > >> > > > should be primary, with the sync version build on top. > > > > >> > > > 2. In languages with proper async support (async-await, > etc.), > > > we > > > > >> can > > > > >> > > skip > > > > >> > > > sync API altogether. As an example of this, you can look at > > the > > > > >> first > > > > >> > > > version of the .NET client [1]. It exposes only async > methods, > > > and > > > > >> it > > > > >> > > > doesn't look like sync counterparts are really needed. > > > > >> > > > 3. In Java (as well as other languages where applicable), we > > > will > > > > >> add > > > > >> > > sync > > > > >> > > > APIs that simply delegate to async APIs. This will help > users > > to > > > > >> avoid > > > > >> > > > CompletableFuture if they don't want to use it. > > > > >> > > > > > > > >> > > > [1] https://github.com/apache/ignite-3/pull/306 > > > > >> > > > > > > > >> > > > Please share your thoughts. > > > > >> > > > > > > > >> > > > -Val > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > -- > Sincerely yours, Ivan Daschinskiy >
