Hi Ee Peng,
On Mon, 2019-02-18 at 08:09 +0000, Yeoh, Ee Peng wrote:
> I did some testing with the latest from resulttool: Update to use
> gitarchive library function.
> http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=rpurdie/t222&id=b9eecaabe56db5bcafff31e67cdabadc42e2d2e4
>
> I had 2 questions.
> 1. For "resulttool regression", currently it was comparing result id
> set without comprehending the difference in the host distro used to
> executed the oeselftest. Example: it was matching oeselftest run with
> fedora28 host distro with oeselftest run with ubuntu18 host distro,
> is this the expected behavior?
> Match: oeselftest_fedora-28_qemux86-64_20190201181656
> oeselftest_ubuntu-18.04_qemux86-64_20190201175023
> Match: oeselftest_fedora-26_qemux86-64_20190131144317
> oeselftest_fedora-26_qemux86-64_20190131144317
> Match: oeselftest_ubuntu-18.04_qemux86-64_20190201175023
> oeselftest_fedora-28_qemux86-64_20190201181656
> Match: oeselftest_opensuse-42.3_qemux86-64_20190126152612
> oeselftest_opensuse-42.3_qemux86-64_20190126152612
There were two reasons for this:
a) the results of the selftest should be independent of which
HOST_DISTRO they're run on so they can be compared.
b) some builds only have one oe-selftest (a-quick) and some have four
(a-full). In an a-quick build, the HOST_DISTRO would likely therefore
be different between two builds but we still would like the tool to
compare them.
> 2. For "resulttool store", I had noticed that it will now generally
> stored testresults.json in a meaningful file directory structure
> based on the store_map except oeselftest. oeselftest currently store
> multiple result id set inside oselftest file directory without
> comprehend the host distro.
>
> For example runtime, store testresult.json with the configured
> store_map.
> ├── oeselftest
> │ └── testresults.json
> ├── runtime
> │ ├── poky
> │ │ ├── qemuarm
> │ │ │ ├── core-image-minimal
> │ │ │ │ └── testresults.json
> │ │ │ ├── core-image-sato
> │ │ │ │ └── testresults.json
> │ │ │ └── core-image-sato-sdk
> │ │ │ └── testresults.json
> │ │ ├── qemuarm64
> │ │ │ ├── core-image-minimal
> │ │ │ │ └── testresults.json
> │ │ │ ├── core-image-sato
> │ │ │ │ └── testresults.json
> │ │ │ └── core-image-sato-sdk
> │ │ │ └── testresults.json
>
> I believe that we shall again comprehend the 'HOST_DISTRO'
> configuration inside the store_map.
> store_map = {
> - "oeselftest": ['TEST_TYPE'],
> + "oeselftest": ['TEST_TYPE','HOST_DISTRO'],
> "runtime": ['TEST_TYPE', 'DISTRO', 'MACHINE', 'IMAGE_BASENAME'],
> "sdk": ['TEST_TYPE', 'MACHINE', 'SDKMACHINE', 'IMAGE_BASENAME'],
> "sdkext": ['TEST_TYPE', 'MACHINE', 'SDKMACHINE',
> 'IMAGE_BASENAME']
>
> Doing so, it will store oeselftest in a more useful file directory
> structure with host distro comprehended.
> └── oeselftest
> ├── fedora-26
> │ └── testresults.json
> ├── fedora-28
> │ └── testresults.json
> ├── opensuse-42.3
> │ └── testresults.json
> └── ubuntu-18.04
> └── testresults.json
The reasoning is the same as the above, its more useful to allow the
files to be directly compared between different host distros.
Cheers,
Richard
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
