Thanks Enrico for the feedback. I would like to merge the PR to complete the setup and reach more feedback for other contributors. Once we complete the setup, the Codecov bot will add a comment on every new pull request
Nicolò Boschi Il giorno mar 22 mar 2022 alle ore 18:32 Enrico Olivelli < eolive...@gmail.com> ha scritto: > Nicolò, > Great stuff indeed. > Some comments inline below > > Thanks > > > Il Mer 16 Mar 2022, 16:45 Nicolò Boschi <boschi1...@gmail.com> ha scritto: > > > Dear community, > > > > Currently we don't have an aggregate code coverage report. There's a > Jacoco > > configuration to generate the report after unit tests but it is not > > easily readable and understandable. > > > > Having a better system would bring a couple of interesting aspects: > > 1. We can block a pull request if the code coverage decreases - that > means > > there are no tests for the new code. > > > > I am afraid that this would lead to some automatic excessive nitpicking. > Let's keep this in the 'future works' basket > > 2. We can have an overview for each modules and discover which features are > > less covered and add new tests > > > > I would like to introduce a new tool that reads, aggregates, stores and > > reports the code coverage for the whole codebase. This tool is Codecov ( > > https://about.codecov.io/) that is one of the most popular tools for > open > > source projects. > > It brings multiple advantages: > > 1. It aggregates all the Jacoco reports in a single report > > 2. It keeps the coverage history > > 3. It is well integrated with Github > > > > The idea is the following: > > - Add this profiling in the CI - for each test suite and upload the > results > > to codecov.io > > - Enable the codecov report that add a comment in the pull > > > > > > Implementation pull: https://github.com/apache/pulsar/pull/14600 > > Codecov comment in the PR: > > https://github.com/apache/pulsar/pull/14600#issuecomment-1061636495 > > Codecov full report: > > > > > https://codecov.io/gh/apache/pulsar/tree/0fa212d40b2bf166b20f3155c40fdb9624f577ed > > > > Initially there were some doubts around whether the jacoco agent would > have > > added a significant overhead (execution time + memory usage). I've seen > > (and you can see the checks times in my pull) that it is not relevant in > > the Pulsar codebase. > > > > The pull only adds Jacoco coverage for unit tests. Integrating jacoco > with > > the integration tests (inside docker) will be more complex but it is > > possible. We can do it in a second moment. > > You basically need to add the jacoco agent inside the docker container, > > enabling it in the java command and downloading the report files before > the > > container stops. > > > > > > Note: Codecov is free for open source projects ( > > https://about.codecov.io/pricing/). It is required to install the app on > > Github Marketplace (https://github.com/marketplace/codecov). > > > > Is it something we should ask to ASF infra probably. > > The PMC can deal with this > > Enrico > > > > BR, > > Nicolò Boschi > > >
