In my opinion, internal consistency is part of the mental model, so
inconsistency reflects a flaw in the model. That said, I think that's probably
talking past you on this a bit, and I get what your point is: ultimately if it
is reasonably intuitive, consistency can be allowed to fall by the wayside a
bit. I guess where I disagree is that I'm not sure this will intuitive for
someone not already steeped in the language. The point I was getting at by
comparing `for` and `with` is that they both make use of the same `<-` operator
in a way that is consistent across both forms, but with `for!` that falls apart.
Now back to `for!`. Even though it looks just like `for`, the `<-` operator
starts to behave like `=`. If you are skimming code and happen to miss the
single character difference between the two (`for` vs `for!`), you will wind up
with a very different idea about what the same code does. The human brain is
terrible at distinguishing small differences like this, it's why you can typo
things like `behavior` and `behaviour` and read right over it without noticing,
sometimes even when you are _trying_ to notice those things.
I think it would be far better for us to use a new operator in place of `<-`,
rather than a new special form that looks basically identical to an existing
one, but works differently in subtle ways. Not to mention, the operator
approach would allow one to mix both `<-` and the new operator together in the
same `for`, should it be useful to do so. In any case, I don't really have a
strong opinion on what that operator is specifically, but I am much more in
favor of that direction, than I am `for!`.
Paul
On Fri, Jun 11, 2021, at 9:24 AM, Adam Lancaster wrote:
> I'm definitely sympathetic to that idea.
>
> I think part of what internal consistency buys us is predictability and
> therefore a quicker path to a good mental model about what the code is going
> to do. But the mental model is the more important thing. Which is just to say
> if we don't have internal consistency but we can get to a good mental model,
> then I think it might be okay.
>
> I think given other functions that follow the same idea, seeing a `for!`
> would certainly communicate "right this is expected to raise under some
> condition" - at least to me.
>
> There's also not an obvious way to have `for` mimic `with` when I think about
> it because say you do this:
>
> ```
> for [a, _] = [1, 2], do: ...
> ```
>
> there is no way to distinguish it from a filter - where `=` should not raise
> a match error.
>
> I think you'd have to more clearly de-mark the difference between the
> generators and the filters, which feels like a big change.
>
>
> Best
>
> Adam
>
>
>
>
>> On 11 Jun 2021, at 00:13, Paul Schoenfelder <[email protected]>
>> wrote:
>>
>> I’m generally in favor of the option to have stricter semantics, but to me
>> the introduction of `for!` feels out of sync with other special forms, none
>> of which are bang-form. Furthermore, especially in contrast to `with`, you
>> end up with this weird dichotomy with the `<-` operator, where sometimes it
>> means a filtering match, and other times where it means strict match. That
>> kind of syntactical inconsistency in a language feels like a bad precedent
>> to set, despite what feels like a reasonable compromise. It’s also notable
>> to me that there are easy ways to program defensively to force match errors
>> if you want them, within the current syntax, but obviously that comes at the
>> cost of more verbosity.
>>
>> I’m not sure what the right answer is, but this feels to me like rushing to
>> solve a specific problem without spending enough time considering how it
>> meshes with the rest of the language in terms of cognitive complexity,
>> particularly for those new to the language.
>>
>> Anyway, that’s my two cents. I’m a fan of the concept for sure, but would
>> almost prefer to see the semantics changed in a major version bump, to match
>> `with`, even if that meant manually updating a bunch of my code, because at
>> least it keeps the language self consistent. I’ll admit I’m probably an
>> outlier on that though.
>>
>> Paul
>>
>> On Thu, Jun 10, 2021, at 6:16 PM, Christopher Keele wrote:
>>> That's fair enough! Though from my perspective both for! and strict: true
>>> would be about equally far from the <- where matches fail. But I can see
>>> the keyword format getting lost in the filters and other keywords.
>>>
>>> On Thu, Jun 10, 2021 at 3:14 PM José Valim <[email protected]> wrote:
>>>> Sorry, I meant to someone reading the code. The strict option is modifying
>>>> the behavior of the operator <-, which may be quite before it in the text.
>>>>
>>>> I prefer for! in this case as it is upfront.
>>>>
>>>> On Fri, Jun 11, 2021 at 00:09 Christopher Keele <[email protected]>
>>>> wrote:
>>>>> > My concern with :strict is that it changes the behavior considerably of
>>>>> > the generators but it may show up only quite later on, far from them,
>>>>> > especially if you have multiple filters.
>>>>>
>>>>> Could you elaborate? I don't quite think I understand, particularly
>>>>> *"[the behaviour] may show up only quite later on"*
>>>>> **
>>>>> Does "quite later" here refer to code distance (the MatchError's
>>>>> stacktrace would point away from/bury the for location)? Or temporal
>>>>> distance?
>>>>>
>>>>> On Thursday, June 10, 2021 at 2:58:03 PM UTC-7 José Valim wrote:
>>>>>> My concern with :strict is that it changes the behavior considerably of
>>>>>> the generators but it may show up only quite later on, far from them,
>>>>>> especially if you have multiple filters.
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 10, 2021 at 23:56 Christopher Keele <[email protected]>
>>>>>> wrote:
>>>>>>> > for {:ok, num} <- list, strict: true, do: num
>>>>>>>
>>>>>>> Agreed, this is more or less exactly what I was pitching.
>>>>>>> On Wednesday, June 9, 2021 at 10:16:25 PM UTC-7 [email protected] wrote:
>>>>>>>> I would like to add a solution within the existing language:
>>>>>>>>
>>>>>>>>
>>>>>>>> ```elixir
>>>>>>>>
>>>>>>>> > list = [{:ok, 1}, {:ok, 2}, {:error, :fail}, {:ok, 4}]
>>>>>>>> > for el <- list, do: ({:ok, num} = el; num)
>>>>>>>> ** (MatchError) no match of right hand side value: {:error, :fail}
>>>>>>>> ```
>>>>>>>> I think this is reasonable.
>>>>>>>>
>>>>>>>> Acctually the built in filtering in `for` caught me off guard, I was
>>>>>>>> expecting for to fail unless all elements matched. So for me the
>>>>>>>> better solution would be to always make matching in `for` strict. But
>>>>>>>> I guess this is too late now for backwards compatibility. Another
>>>>>>>> alternative to `for!` would be:
>>>>>>>>
>>>>>>>> ```elixir
>>>>>>>>
>>>>>>>> > list = [{:ok, 1}, {:ok, 2}, {:error, :fail}, {:ok, 4}]
>>>>>>>> > for {:ok, num} <- list, strict: true, do: num
>>>>>>>> ** (MatchError) no match of right hand side value: {:error, :fail}
>>>>>>>> ```
>>>>>>>>
>>>>>>>> I don't like the use of the exclamation mark in `for!` because it has
>>>>>>>> little meaning relative to the existing use of the exclamation mark in
>>>>>>>> Elixir.
>>>>>>>>
>>>>>>>> onsdag 9. juni 2021 kl. 13:17:04 UTC+2 skrev [email protected]:
>>>>>>>>> I also love the proposal.
>>>>>>>>>
>>>>>>>>> It's a shame we can't re-use the `with` semantics of `=` raising a
>>>>>>>>> match error in the for.
>>>>>>>>>
>>>>>>>>> My two cents is `for!` makes the most sense, and follows the
>>>>>>>>> conventions of other functions.
>>>>>>>>>
>>>>>>>>> Best
>>>>>>>>>
>>>>>>>>> Adam
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 8 Jun 2021, at 18:18, Christopher Keele <[email protected]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> This feature would be very useful, I've experience this
>>>>>>>>>> signature-change pain point before too (and kind of have been
>>>>>>>>>> avoiding `for` ever since, TBH).
>>>>>>>>>>
>>>>>>>>>> I'm reluctant to increase the surface area of the language itself,
>>>>>>>>>> what do you think about adding a `:strict` option to `for` instead
>>>>>>>>>> of a new special form/kernel macro/operator?
>>>>>>>>>> On Monday, June 7, 2021 at 9:50:45 AM UTC-7 [email protected]
>>>>>>>>>> wrote:
>>>>>>>>>>> ## Background
>>>>>>>>>>>
>>>>>>>>>>> `for` comprehensions are one of the most powerful features in
>>>>>>>>>>> Elixir. It supports both enumerable and bitstring generators,
>>>>>>>>>>> filters through boolean expressions and pattern matching,
>>>>>>>>>>> collectibles with `:into` and folding with `:reduce`.
>>>>>>>>>>>
>>>>>>>>>>> One of the features are automatic filtering by patterns in
>>>>>>>>>>> generators:
>>>>>>>>>>>
>>>>>>>>>>> ```elixir
>>>>>>>>>>> list = [{:ok, 1}, {:ok, 2}, {:error, :fail}, {:ok, 4}]
>>>>>>>>>>> for {:ok, num} <- list, do: num
>>>>>>>>>>> #=> [1, 2, 4]
>>>>>>>>>>> ```
>>>>>>>>>>>
>>>>>>>>>>> Generator filtering is very powerful because it allows you to
>>>>>>>>>>> succinctly filter out data that is not relevant to the
>>>>>>>>>>> comprehension in the same expression that you are generating
>>>>>>>>>>> elements out of your enumerable/bitstrings. But the implicit
>>>>>>>>>>> filtering can be dangerous because changes in the shape of the data
>>>>>>>>>>> will silently be removed which can cause hard to catch bugs.
>>>>>>>>>>>
>>>>>>>>>>> The following example can show how this can be an issue when
>>>>>>>>>>> testing `Posts.create/0`. If a change causes the function to start
>>>>>>>>>>> returning `{:ok, %Post{}}` instead of the expected `%Post{}` the
>>>>>>>>>>> test will pass even though we have a bug.
>>>>>>>>>>>
>>>>>>>>>>> ```elixir
>>>>>>>>>>> test "create posts" do
>>>>>>>>>>> posts = Posts.create()
>>>>>>>>>>> for %Post{id: id} <- posts, do: assert is_integer(id)
>>>>>>>>>>> end
>>>>>>>>>>> ```
>>>>>>>>>>>
>>>>>>>>>>> The example uses a test to highlight the issue but it can just as
>>>>>>>>>>> well happen in production code, specially when refactoring in other
>>>>>>>>>>> parts of the code base than the comprehension.
>>>>>>>>>>>
>>>>>>>>>>> Elixir is a dynamically typed language but dynamic typing errors
>>>>>>>>>>> are less of an issue compared to many other dynamic languages
>>>>>>>>>>> because we are usual strict in the data we accept by using pattern
>>>>>>>>>>> matching and guard functions. `for` is by design not strict on the
>>>>>>>>>>> shape of data it accepts and therefor loses the nice property of
>>>>>>>>>>> early failure on incorrect data.
>>>>>>>>>>>
>>>>>>>>>>> ## Proposal
>>>>>>>>>>>
>>>>>>>>>>> I propose an alternative comprehension macro called `for!` that has
>>>>>>>>>>> the same functionality as `for` but instead of filtering on
>>>>>>>>>>> patterns in generators it will raise a `MatchError`.
>>>>>>>>>>>
>>>>>>>>>>> ```elixir
>>>>>>>>>>> posts = [{:ok, %Post{}}]
>>>>>>>>>>> for! %Post{id: id} <- posts, do: assert is_integer(id)
>>>>>>>>>>> #=> ** (MatchError) no match of right hand side value: {:ok,
>>>>>>>>>>> %Post{}}
>>>>>>>>>>> ```
>>>>>>>>>>>
>>>>>>>>>>> Pattern matching when not generating values with `=` remains
>>>>>>>>>>> unchanged.
>>>>>>>>>>>
>>>>>>>>>>> `for!` gives the developer an option to be strict on the data it
>>>>>>>>>>> accepts instead of silently ignoring data that does not match.
>>>>>>>>>>>
>>>>>>>>>>> ## Other considerations
>>>>>>>>>>>
>>>>>>>>>>> You can get strict matching with `for` today by first assigning to
>>>>>>>>>>> a variable. This way you can also mix filtering and strict matching
>>>>>>>>>>> patterns.
>>>>>>>>>>>
>>>>>>>>>>> ```elixir
>>>>>>>>>>> posts = [{:ok, %Post{}}]
>>>>>>>>>>> for post <- posts,
>>>>>>>>>>> %Post{id: id} = post,
>>>>>>>>>>> do: assert is_integer(id)
>>>>>>>>>>> #=> ** (MatchError) no match of right hand side value: {:ok,
>>>>>>>>>>> %Post{}}
>>>>>>>>>>> ```
>>>>>>>>>>>
>>>>>>>>>>> Another alternative is to introduce a new operator such as `<<-`
>>>>>>>>>>> (the actual token can be anything, `<<-` is only used as an
>>>>>>>>>>> example) for raising pattern matches instead of introducing a
>>>>>>>>>>> completely new macro.
>>>>>>>>>>>
>>>>>>>>>>> ```elixir
>>>>>>>>>>> posts = [{:ok, %Post{}}]
>>>>>>>>>>> for %Post{id: id} <<- posts, do: assert is_integer(id)
>>>>>>>>>>> #=> ** (MatchError) no match of right hand side value: {:ok,
>>>>>>>>>>> %Post{}}
>>>>>>>>>>> ```
>>>>>>>>>>>
>>>>>>>>>>> A downside of adding new functions or macros is that it doesn't
>>>>>>>>>>> compose as well compared to adding options (or operators) to
>>>>>>>>>>> existing functions. If we want to add another variant of
>>>>>>>>>>> comprehensions in the future we might be in the position that we
>>>>>>>>>>> need 4 macros, and then 8 and so on.
>>>>>>>>>>>
>>>>>>>>>>> Another benefit of adding an operator is that you can mix both `<-`
>>>>>>>>>>> and `<<-` in a single comprehension.
>>>>>>>>>>>
>>>>>>>>>>> The downside of an operator is that it adds more complexity for the
>>>>>>>>>>> language user. We would also need an operator that is visually
>>>>>>>>>>> close to `<-` but still distinctive enough that they are easy to
>>>>>>>>>>> separate since their behavior are very difference.
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> You received this message because you are subscribed to the Google
>>>>>>>>>> Groups "elixir-lang-core" group.
>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>>>>> send an email to [email protected].
>>>>>>>>>> To view this discussion on the web visit
>>>>>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/42adcfba-12d8-4469-a156-f412b0d290a9n%40googlegroups.com
>>>>>>>>>>
>>>>>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/42adcfba-12d8-4469-a156-f412b0d290a9n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>>>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "elixir-lang-core" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>>>> an email to [email protected].
>>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/d/msgid/elixir-lang-core/f4d5c0be-567a-4a7d-9b39-68202226c788n%40googlegroups.com
>>>>>>>
>>>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/f4d5c0be-567a-4a7d-9b39-68202226c788n%40googlegroups.com?utm_medium=email&utm_source=footer>.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google Groups
>>>>> "elixir-lang-core" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send an
>>>>> email to [email protected].
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/elixir-lang-core/0ce03abc-61bb-4423-b6a8-704d1d62169fn%40googlegroups.com
>>>>>
>>>>> <https://groups.google.com/d/msgid/elixir-lang-core/0ce03abc-61bb-4423-b6a8-704d1d62169fn%40googlegroups.com?utm_medium=email&utm_source=footer>.
>>>>
>>>> --
>>>> You received this message because you are subscribed to a topic in the
>>>> Google Groups "elixir-lang-core" group.
>>>> To unsubscribe from this topic, visit
>>>> https://groups.google.com/d/topic/elixir-lang-core/LEUD2alHPiE/unsubscribe.
>>>> To unsubscribe from this group and all its topics, send an email to
>>>> [email protected].
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4K01hBRkjLaRPj5ktViNNjYqdFbKdysvFcDVG%3DgBp78dA%40mail.gmail.com
>>>>
>>>> <https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4K01hBRkjLaRPj5ktViNNjYqdFbKdysvFcDVG%3DgBp78dA%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "elixir-lang-core" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an
>>> email to [email protected].
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/elixir-lang-core/CAD9kT2QPn_prFiS%2BR9eemqA43DMvvOB8NrAweL2PgE_ZR2g6Cg%40mail.gmail.com
>>>
>>> <https://groups.google.com/d/msgid/elixir-lang-core/CAD9kT2QPn_prFiS%2BR9eemqA43DMvvOB8NrAweL2PgE_ZR2g6Cg%40mail.gmail.com?utm_medium=email&utm_source=footer>.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "elixir-lang-core" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/elixir-lang-core/8a6cf634-cda5-4445-8230-4b7b69ed5ca8%40www.fastmail.com
>>
>> <https://groups.google.com/d/msgid/elixir-lang-core/8a6cf634-cda5-4445-8230-4b7b69ed5ca8%40www.fastmail.com?utm_medium=email&utm_source=footer>.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/65609056-4A25-45FA-B91F-84D4DF292129%40a-corp.co.uk
>
> <https://groups.google.com/d/msgid/elixir-lang-core/65609056-4A25-45FA-B91F-84D4DF292129%40a-corp.co.uk?utm_medium=email&utm_source=footer>.
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/4f04033a-d509-460e-8205-ad23e1251b1e%40www.fastmail.com.
