[
https://issues.jenkins-ci.org/browse/JENKINS-12810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=159557#comment-159557
]
Bruno P. Kinoshita commented on JENKINS-12810:
----------------------------------------------
Hi Jaroslavas! Thanks for confirming that this issue is happening indeed.
I haven't had time to thoroughly investigate what is happening, but I have a
hunch that it has something to do with the result seeking strategy. I think you
are using JUnit to execute your files. It generates XML's, that the plug-in
looks for after it executed the tests.
The problem may be happening if you have more than one test class, test name or
suite name that collides with the key custom field value. That happens because
if you provided a pattern to find JUnit XML files, the plug-in will use all
JUnit strategies to seek test results. I.e.: say you have a test case in
testlink which the key custom field value is 'TestOperationA', and you have a
JUnit class with name TestOperationA and a suite with name TestOperationA too.
The JUnit class is included in this suite. What will happen, is that the
plug-in will find these both items (the class and the suite) and for each one
will attach its corresponding XML file.
> 2 test methods - report get attached twice!
Depends on the way the XML is generated, the key custom field value, name of
test class, test suite. At moment, the plug-in does not work with test methods.
Now the good news.
1. Test methods will be supported in version 3.1
2. For three weeks I've been refactoring the plug-in code (to simplify the
code) and add an Extension point (Jenkins API) for Result Seeking Strategy.
3. Now you will be able to choose which strategies you want to use, so you can
choose and say: In this job, the plug-in should use only test methods. It will
reduce the execution time, as well as will avoid wrong attachments / execution
status updates.
4. A result strategy now has options, like the pattern for report files, the
key custom field name (it's per strategy now), and whether it will upload
attachments or not
> Is it possible to prevent jenkins from attaching reports at all?
It's already coded in the code trunk, but I still have to test the attachment
under several circumstances to ensure that I didn't break anything with the
refactoring.
Would you like to test an 3.1 alpha version? I can finish testing the upload of
attachments in one or two days.
All the best!
Bruno
> JUnit test results are getting wrongly attached to all the test cases
> ---------------------------------------------------------------------
>
> Key: JENKINS-12810
> URL: https://issues.jenkins-ci.org/browse/JENKINS-12810
> Project: Jenkins
> Issue Type: Bug
> Components: testlink
> Affects Versions: current
> Reporter: Vignesh Senapathy
> Assignee: Bruno P. Kinoshita
> Fix For: current
>
> Attachments: 1.jpg, 2.jpg, 3.jpg, last_tc.png, tc1.png, tc2.png
>
>
> Hi,
> I am using the Testlink Jenkins plugin and when i run my jobs in Jenkins the
> Junit Test results are getting generated. This in turn calls the Testlink xml
> rpc and attaches the results to the test case, but the test results are
> attached wrongly. The 1st test case has 6 results, the next has 5 and so on
> and the last one has 1 test result attached to it. I dont know if it is bug
> which is causing this. Please let me know what seems to be the problems. Also
> the execution flags are correctly rendered in this. which ever test case
> fails is marked as failed and which passed is marked as passed.
> Also I have one custom field which maps to the test suite name and test
> classname.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
