Hello again, thanks for your time and prompt reply! I am not sure, I might not have been clear enough. For example, given an umbrella project, I can run `mix test --export-coverage` from the project root. Even if `:covertool` is specified as a tool in `test_coverage:`, the .coverdata files are exported correctly alongside XML within each folder.
The problem is: when I run `mix test.coverage`, it takes all .coverdata files and compiles them into an HTML report <https://github.com/elixir-lang/elixir/blob/master/lib/mix/lib/mix/tasks/test.coverage.ex#L179> without any consideration for the specified coverage tool. Yes, this issue could be solved with a custom command within the package to operate upon coverdata files, but I still believe that this behavior is counterintuitive. Maybe another option could be specified explicitly for report generation based on multiple coverdata files? One more example (although not my use case) could be the `--partitions` flag. It implicitly generates coverdata files, which are then expected to be compiled via `mix test.coverage`. So if a person has the tool specified in `test_coverage`, then running `mix test` will produce different reports from `mix test --partitions` and `mix test.coverage` because one will use the coverage tool and the other will not. Thank you again for your attention! On Thursday, 12 November 2020 at 11:39:02 UTC+3 José Valim wrote: > Hi Ilya, thanks for the report! > > mix test.coverage is dependent on how the default coverage tool exports > data. We cannot aggregate coverage reports from random tools because we > don't know how these tools export their results. Therefore, it is up to > these tools to do two changes: > > 1. Respect the :export option in their coverage tool > 2. Provide their own mix task that aggregate the results > > So please reach out to them and consider sending pull requests so they > make this feature available, > > Thank you! > > On Wed, Nov 11, 2020 at 10:59 PM Ilya Borovitinov <[email protected]> > wrote: > >> Hello! >> >> Today I encountered a problem when trying to set up testing + coverage >> for the new Elixir project with Elixir 1.11. >> >> We are using Gitlab as our source control and CI, and Gitlab can display >> covered lines if you were to provide coverage in the Cobertura XML format. >> That's all good, and Elixir/Erlang ecosystem even has `covertool` >> <https://github.com/covertool/covertool> package for this purpose. >> >> I have an umbrella project with 3 apps currently, and so I read in the >> release notes that I could use `mix test` with `--export-coverage` flag to >> collect coverage from each app and then compile them with `mix >> test.coverage`. That's awesome. >> >> Here's the problem though: `mix test` calls upon the specified tool to >> process coverage data, but `mix test.coverage` just generates HTML, so we >> cannot use any tools to properly compile and process coverage data across >> all apps (or partitions). Basically, I feel that there shouldn't be such a >> discrepancy in usage between these two commands. >> >> With great admiration for the language, >> Ilya Borovitinov >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elixir-lang-core" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elixir-lang-core/848b4145-ef5e-48fa-863f-d0fb65f2a8cfn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/elixir-lang-core/848b4145-ef5e-48fa-863f-d0fb65f2a8cfn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "elixir-lang-core" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-core/9ceb51e7-82cf-4edb-8315-cd05df5a1bbfn%40googlegroups.com.
