20/05/2018 19:41, Parthasarathy, JananeeX M:
> From: Thomas Monjalon [mailto:tho...@monjalon.net]
> > 18/05/2018 15:32, Parthasarathy, JananeeX M:
> > > From: Thomas Monjalon [mailto:tho...@monjalon.net]
> > > > 15/05/2018 18:14, Parthasarathy, JananeeX M:
> > > > > From: Thomas Monjalon [mailto:tho...@monjalon.net]
> > > >
> > > > I think we need to keep a fast subset of tests.
> > > >
> > > > Please check in mk/rte.sdktest.mk:
> > > > PERFLIST=ring_perf,mempool_perf,memcpy_perf,hash_perf,ti
> > > > mer_perf
> > > >
> > > > test-fast: BLACKLIST=-$(PERFLIST)
> > >
> > > We have added the below test cases to blacklist in mk/rte.sdktest.mk
> > > reciprocal_division,reciprocal_division_perf,lpm_perf,red_all
> > >
> > > Now time taken is around 4mins.
> > > Please let us know is there any minimum time limit for test execution.
> > > Is this time duration fine?
> >
> > 4 minutes, it's a bit long for developer oriented tests.
> >
> > We need to distinguish which tests are enough for quick tests,
> > and which ones can be in another category for full validation purpose.
> >
> > Please can you make a summary of which tests you are adding,
> > and propose a priority classification.
> >
> > Thanks
>
> Please find below the test cases newly added to sanity script
> (autotest_data.py) with priority classification.
>
> P1 - Main test cases
We can move some tests from P1. Examples:
- bonding can be P2 with other drivers.
- PMD perf and ring perf can P3 with other perf tests.
> P2 - Cryptodev test cases which requires dependent libraries (Can be added
> if the environment is set accordingly)
> P3 - Perf test cases and others which takes more time (> 10s) (these will be
> in the blacklist test cases)
> P4 - Logging/Dump test cases (to be checked whether these are really
> necessary)
I think this classification is good.
We can adjust the categories in the makefile to reflect it.
The goal is to have a really quick and core subset (equal P1 category),
and other categories available with a different make command
(drivers for P2, perf for P3 and debug for P4).
> The blacklist is updated in makefile "rte.sdktest.mk" with below list
> of test cases to reduce the total test execution time.
>
> Blacklist test cases are
> ring_perf,mempool_perf,memcpy_perf,hash_perf,timer_perf,
> reciprocal_division,reciprocal_division_perf,lpm_perf,red_all,barrier,
> hash_multiwriter,timer_racecond,efd,hash_functions,eventdev_selftest_sw,
> member_perf,efd_perf,lpm6_perf,red_perf,distributor_perf
>
> Currently after the above blacklist update, the total execution time of "make
> test-fast" is 54s.
Less than a minute is a good limit!
Maybe we can do even better with above suggestions.
Thanks for working on it.
Feel free to send a new patch when you think it is ready.
We can try to have the new autotests when starting 18.08.
Then we should communicate to enforce a better coverage of the autotests.
