I've finally read it as well :) Comments below.
> At the moment, that task yaml file for rpmlint [1] looks like:
>
> dependencies:
> - rpmlint
> - libtaskbot
>
> input:
> args: envr,arch
These arguments are provided by the event trigger, right? So, the event trigger
offers a set of arguments and we pick some of them inside "input" section; they
can be then used in other sections. Is that right? ('arch' is not actually used
anywhere in this demo file).
>
> preparation:
> koji: download $envr
This is great, a preparation phase is exactly what I wanted.
>
> execution:
> python: run_rpmlint.py $workdir
1) You don't actually use $workdir in run_rpmlint.py. I guess that's a bug? Or
will we guarantee that CWD is set to $workdir?
2) Can we have multiple execution lines that are run sequentially (or in
parallel?) and then consider the aggregate result as a test result? One example
is in our outdated rats_install test:
for ks in ['fedora','updates','updates-testing']:
job.run_test('rats_install', tag=ks, ks=ks, **autoqa_args)
We can of course define this as three separate tests, but sometimes running a
few different variants together in a single test run might make sense.
>
> post:
> shell: clean $workdir
How is this supposed to work? 'clean' is not an existing command, is that a
keyword?
Will we support running arbitrary shell commands or custom scripts (e.g.
'./myscript') in this or other phases (like preparation)?
>
> report:
> resultdb: something
I guess it's possible to run and report multiple results from a single test
run? I.e. upgradepath checking 20 builds and reporting 20 separate test results.
Can I also report a single result from several execution lines?
I know this is just a demo and we will probably need to discover the answers
for many of the questions above collectively. I was trying to think about
different use cases a maybe discover some missing ones.
_______________________________________________
qa-devel mailing list
qa-de...@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/qa-devel
