Good call!
What if we simply convert any line where = is the root expression and the
left-side is not a variable to an assertion?
The rationale is that this is an ok doctest:
iex> {:error, _} = do_something(var, 0)
iex> :ok = do_something(var, 1)
And we would most likely want to raise on these cases too.
So we can just convert everything automatically whenever we can.
*José Valim*
www.plataformatec.com.br
Founder and Director of R&D
On Wed, Nov 27, 2019 at 11:22 AM Devon Estes <[email protected]>
wrote:
> I think doctests are extremely helpful and powerful since they encourage
> developers to write testable, accurate documentation for their code. But
> the lack of some of the niceties that exist in ExUnit might make it less
> likely for developers to choose doctests instead of ExUnit tests, so I've
> been thinking of ways to narrow this gap.
>
> Currently if we are writing doctests and want to assert that the output of
> the function we're testing matches a pattern instead of testing the output
> for strict equality, we need to do one of the following two things:
>
> ```
> iex> match?(%{c: _}, Map.put(%{a: :b}, :c, :d))
> true
>
> iex> %{c: _} = Map.put(%{a: :b}, :c, :d)
> %{a: :b, c: :d}
> ```
>
> There are a few problems I see with this at the moment. First is the
> failure messages - they're not nearly as good as the failure messages in
> ExUnit at the moment for pattern matching.
>
> ```
> iex> %{f: _} = Map.put(%{a: :b}, :c, :d)
> %{a: :b, c: :d}
> ```
>
> 1) doctest Benchee.Configuration.init/1 (1) (Benchee.ConfigurationTest)
> test/benchee/configuration_test.exs:3
> ** (MatchError) no match of right hand side value: %{a: :b, c: :d}
> stacktrace:
> (for doctest at) lib/benchee/configuration.ex:167: (test)
>
>
> And then for the other one we get an even less helpful error message:
>
> ```
> iex> match?(%{f: _}, Map.put(%{a: :b}, :c, :d))
> true
> ```
>
> 1) doctest Benchee.Configuration.init/1 (1) (Benchee.ConfigurationTest)
> test/benchee/configuration_test.exs:3
> Doctest failed
> doctest:
> iex> match?(%{f: _}, Map.put(%{a: :b}, :c, :d))
> true
> code: match?(%{f: _}, Map.put(%{a: :b}, :c, :d)) === true
> left: false
> right: true
> stacktrace:
> lib/benchee/configuration.ex:167: Benchee.Configuration (module)
>
> To get the very nice new colored diff for matches available in 1.10 in our
> failure message we can do the following:
>
> ```
> iex> assert %{f: _} = Map.put(%{a: :b}, :c, :d)
> %{a: :b, c: :d}
> ```
>
> 1) doctest Benchee.Configuration.init/1 (1) (Benchee.ConfigurationTest)
> test/benchee/configuration_test.exs:3
> match (=) failed
> code: assert %{f: _} = Map.put(%{a: :b}, :c, :d)
> left: %{f: _}
> right: %{a: :b, c: :d}
> stacktrace:
> (for doctest at) lib/benchee/configuration.ex:167: (test)
>
> But putting this assertion in a doctest takes away from value of the
> example as documentation since the example is no longer really able to be
> cut and pasted - it's now more test than documentation and doesn't really
> belong here.
>
> Also, If you want to pattern match in the final line of a doctest, you
> still need to put the return value of the match (which is always the value
> being matched against) at the end of the test or else it's not parsed as a
> valid test, which makes the pattern match useless since you're already
> making the more strict comparison with `===`. Basically, we can't do this
> today:
>
> ```
> iex> %{f: _} = Map.put(%{a: :b}, :c, :d)
>
> iex> %{e: _} = Map.put(%{a: :b}, :e, :f)
>
> iex> %{f: _} = Map.put(%{a: :b}, :f, :e)
> ```
>
> My proposal is that we allow this to be a valid doctest, and that we use
> it to give really good failure messages. Anytime the last line of a doctest
> is a pattern match, we convert that into `assert pattern = expr` when
> parsing and running the test.
>
> The other thing I came up with (that I don't particularly like but figured
> I'd put it out there just in case someone builds something good off of it)
> is to use the anonymous function shorthand syntax to represent the output
> of the previous line and do something like this:
>
> ```
> iex> Map.put(%{a: :b}, :c, :d)
> %{f: _} = &1
>
> iex> Map.put(%{a: :b}, :e, :f)
> %{e: _} = &1
> ```
>
> This gets a little further away from the intended purpose of documentation
> and towards more of a test, but it would still give a better failure
> message for developers and keeps the requirement that the last line of a
> doctest includes something to test against.
>
> If folks have any better ideas for this I'd love to hear them!
>
> --
> You received this message because you are subscribed to the Google Groups
> "elixir-lang-core" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elixir-lang-core/b93ea3e4-9ebe-4090-b14c-bae138c60f4f%40googlegroups.com
> <https://groups.google.com/d/msgid/elixir-lang-core/b93ea3e4-9ebe-4090-b14c-bae138c60f4f%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"elixir-lang-core" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elixir-lang-core/CAGnRm4K7fJ45BH22Pp2tBAYJnaO3TWRchdz5KzS__h1arJ2UhQ%40mail.gmail.com.
