I have been thinking more about this problem and there is still one issue
for us to solve: when you say you are waiting for "n" tasks, does it mean
you are waiting for "n" successes or "n" results (regardless if they are
success or failure)?
So I thought perhaps a better solution is to introduce something called
Task.yield_stream/2. "yield_stream" receives an enumerable of tasks and
emits them as they complete. To get the first successful task you would do:
tasks
|> Task.yield_stream(ordered: false, timeout: 5000)
|> Stream.filter(&match?({:ok, _}))
|> Enum.at(0)
And this made me wonder: is there any reason why you can't use
Task.async_stream?
On Thu, Apr 1, 2021 at 3:59 PM José Valim <[email protected]> wrote:
> I don't see a reason why we wouldn't return the same order. You can easily
> get the ones you want by `for {task, {:ok, value}} <- result do`. Plus
> forcing people to iterate will remind them that they most likely need to
> shutdown the other tasks.
>
> On Thu, Apr 1, 2021 at 3:57 PM [email protected] <
> [email protected]> wrote:
>
>> > In this sense, yield_many becomes a special case of yield_first where
>> you want to yield on all given tasks.
>>
>> I'm not sure if we would want that, because in yield_many the returned
>> list will be in the same order as the tasks supplied in the tasks input
>> argument.
>> However yield_first to preserve the semantics of "returning as soon it
>> finishes", maybe we should return the tasks in the order of finish (the
>> first to complete first in the list).
>> That makes sense?
>>
>> Em quarta-feira, 31 de março de 2021 às 19:35:47 UTC-3,
>> [email protected] escreveu:
>>
>>> +1 for yield_first(tasks, n, timeout)
>>>
>>> It seems to better convey the meaning "yield the first n tasks".
>>>
>>> Em qua., 31 de mar. de 2021 às 19:26, Felipe Stival <[email protected]>
>>> escreveu:
>>>
>>>> +1 for yield_first(tasks, n, timeout)
>>>>
>>>>
>>>> On Thu, Apr 1, 2021, 01:11 José Valim <[email protected]> wrote:
>>>>
>>>>> It was not possible to implement yield_first in Elixir but now that we
>>>>> have map_get in guards, we can easily do so by putting all refs in a map
>>>>> and only getting messages from the inbox where the ref is in the map. The
>>>>> number of tasks to wait and the maximum timeout should be configurable
>>>>> too.
>>>>> For example:
>>>>>
>>>>> yield_first(task, 3, 1000)
>>>>>
>>>>> The above will yield the first 3 tasks within 1000ms. It should have
>>>>> the same result type as yield_many. In this sense, yield_many becomes a
>>>>> special case of yield_first where you want to yield on all given tasks.
>>>>>
>>>>> Another option is to not introduce a new function but instead
>>>>> introduce a new argument to yield_many with the limit to yield:
>>>>>
>>>>> yield_many(task, 1000, 3)
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Mar 31, 2021 at 11:52 PM [email protected] <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> I think the proposal would work differently from yield_many.
>>>>>> yield_many "receives a list of tasks and waits for their replies in the
>>>>>> given time interval".
>>>>>> The proposal of the new function is to return as soon as the first
>>>>>> task finishes.
>>>>>>
>>>>>> For example, if we start tasks to make call to 3 different remote
>>>>>> APIs, with different response times.
>>>>>>
>>>>>> API A - 50ms
>>>>>> API B - 500ms
>>>>>> API C - 1500ms
>>>>>>
>>>>>> tasks = [
>>>>>> Task.async(&api_a/0),
>>>>>> Task.async(&api_b/0),
>>>>>> Task.async(&api_c/0)
>>>>>> ]
>>>>>>
>>>>>> # returns result of API A and API B waiting for 1000ms
>>>>>> Task.yield_many(tasks, 1000)
>>>>>>
>>>>>> With run using `yield_many` we would wait for the 1000ms and get the
>>>>>> responses of API A and API B.
>>>>>>
>>>>>> The proposal of the new function is to return as soon as we get a
>>>>>> response.
>>>>>> Using the same example of the 3 calls, we would wait only for 50ms
>>>>>> (as soon as the first task finishes) and return the result of the first
>>>>>> task finishing, without waiting for the other call.
>>>>>>
>>>>>> tasks = [
>>>>>> Task.async(&api_a/0),
>>>>>> Task.async(&api_b/0),
>>>>>> Task.async(&api_c/0)
>>>>>> ]
>>>>>>
>>>>>> # returns only result of API A waiting for 50ms
>>>>>> Task.proposed_function(tasks)
>>>>>>
>>>>>> Em quarta-feira, 31 de março de 2021 às 18:06:47 UTC-3,
>>>>>> [email protected] escreveu:
>>>>>>
>>>>>>> Check out Task.yield_many/2 (
>>>>>>> https://hexdocs.pm/elixir/Task.html#yield_many/2) :-)
>>>>>>>
>>>>>>> On 31 Mar 2021, at 22:54, [email protected] <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>> *Proposal*
>>>>>>>
>>>>>>> Add a function to the Task module that takes a list of tasks, and
>>>>>>> returns as soon as one of the tasks finishes, shuting down the other
>>>>>>> tasks.
>>>>>>> The behaviour would pretty similar to what Javascript have with
>>>>>>> Promise.any
>>>>>>> https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/any
>>>>>>>
>>>>>>> *Motivation*
>>>>>>>
>>>>>>> One scenario that it could be useful is when we are integrating with
>>>>>>> multiple APIs (providers) of the same data, and we want only the fastest
>>>>>>> result without needing to wait for the other requests to complete.
>>>>>>>
>>>>>>> Today I think this could be implemented with something similar to
>>>>>>> the following code:
>>>>>>>
>>>>>>> tasks = [
>>>>>>> Task.async(&heavy_fun_1/0),
>>>>>>> Task.async(&heavy_fun_2/0),
>>>>>>> Task.async(&heavy_fun_3/0)
>>>>>>> ]
>>>>>>>
>>>>>>> receive do
>>>>>>> {ref, result} ->
>>>>>>> tasks
>>>>>>> |> Enum.reject(fn task -> task.ref == ref end)
>>>>>>> |> Enum.each(&Task.shutdown/1)
>>>>>>>
>>>>>>> result
>>>>>>> after
>>>>>>> 5000 ->
>>>>>>> {:error, :timeout}
>>>>>>> end
>>>>>>>
>>>>>>> However that seems to be a common enough pattern to add to the
>>>>>>> standard library.
>>>>>>>
>>>>>>> *Questions*
>>>>>>>
>>>>>>> - Am I missing something here and this could already be easily
>>>>>>> accomplished with the existing API?
>>>>>>> - What should be the behaviour when the first task to complete
>>>>>>> exits?
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "elixir-lang-core" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to [email protected].
>>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/59a5b5af-528b-4f1f-8e17-6dad9edfe9ccn%40googlegroups.com
>>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/59a5b5af-528b-4f1f-8e17-6dad9edfe9ccn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>>> .
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "elixir-lang-core" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an email to [email protected].
>>>>>> To view this discussion on the web visit
>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/fe8d048f-605b-4b7a-ad2d-64fb11727d4dn%40googlegroups.com
>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/fe8d048f-605b-4b7a-ad2d-64fb11727d4dn%40googlegroups.com?utm_medium=email&utm_source=footer>
>>>>>> .
>>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "elixir-lang-core" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4L%2BogpwDohNaRoZHkT0%3DkOdFROdT37BQJx3k%3D%3DzZqQzAA%40mail.gmail.com
>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4L%2BogpwDohNaRoZHkT0%3DkOdFROdT37BQJx3k%3D%3DzZqQzAA%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>>> .
>>>>>
>>>> --
>>>>
>>> You received this message because you are subscribed to a topic in the
>>>> Google Groups "elixir-lang-core" group.
>>>> To unsubscribe from this topic, visit
>>>> https://groups.google.com/d/topic/elixir-lang-core/ZIFsisK12CM/unsubscribe
>>>> .
>>>> To unsubscribe from this group and all its topics, send an email to
>>>> [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elixir-lang-core/CAKC64%2BwFh20dRTVnJi5QC8Ekk5CNcSx8k8jW_xBP65rOYGukYw%40mail.gmail.com
>>>> <https://groups.google.com/d/msgid/elixir-lang-core/CAKC64%2BwFh20dRTVnJi5QC8Ekk5CNcSx8k8jW_xBP65rOYGukYw%40mail.gmail.com?utm_medium=email&utm_source=footer>
>>>> .
>>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "elixir-lang-core" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elixir-lang-core/c31c31f1-2de8-41a4-a3f3-f5b924c2ea1an%40googlegroups.com
>> <https://groups.google.com/d/msgid/elixir-lang-core/c31c31f1-2de8-41a4-a3f3-f5b924c2ea1an%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4JLeHgiAkqn7nWo34DRc4sh5b1R9aMhb_Y_wfGhOkSY3Q%40mail.gmail.com.
