[I] How to utilize all sessions if tests are run across multiple xmls [maven-surefire]

[via GitHub]

VZEROALL opened a new issue, #842:
URL: https://github.com/apache/maven-surefire/issues/842

   ### New feature, improvement proposal
   
   hello
   
   we use testNG to run tests via suites, where each suite denotes the 
functionality of each team, e.g. team1, team2, team3
   
   The problem is that all suites have different qty of tests. because of this 
the free quota (selenium grid, moon etc) in the cloud is idle. how to maximize 
the use of free sessions?
   
   For example, we run 3 suites in parallel. two suites have 10 tests each, the 
third suite has 20 tests.
   (suppose the total free quota is 15 sessions.)
   The settings are as follows:
   
   `
   <plugin>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-surefire-plugin</artifactId>
     <version>${maven-surefire-plugin.version}</version>
     
                    <configuration>
   
                       <argLine>
                           
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                       </argLine>
   
                       <suiteXmlFiles>
                           <suiteXmlFile>team1.xml</suiteXmlFile>
                           <suiteXmlFile>team2.xml</suiteXmlFile>
                           <suiteXmlFile>team3.xml</suiteXmlFile>
                       </suiteXmlFiles>
   
                       <parallel>tests</parallel>
                       <threadCount>5</threadCount>
                       
                   <properties>
                       <property>
                           <name>suitethreadpoolsize</name>
                           <value>3</value>
                       </property>
                   </properties>
                   
             </configuration>
             
    </plugin>
   
   `
   
   result
   
   
![Image](https://github.com/user-attachments/assets/60d140ea-21af-4e48-b92f-4dfffe15f601)
   
   
![Image](https://github.com/user-attachments/assets/f6214a55-a12a-4b96-b259-253d5b849ea4)
   
   the third suite used 5 sessions. 10 free sessions were not used.
   (I would like the remaining 10 tests in suite 3 to be executed in 10 threads 
in parallel)
   
   
   second example
   
   config
   
   `
               <plugin>
                   <groupId>org.apache.maven.plugins</groupId>
                   <artifactId>maven-surefire-plugin</artifactId>
                   <version>${maven-surefire-plugin.version}</version>
                   <configuration>
   
                       <argLine>
                           
-javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                       </argLine>
   
                       <suiteXmlFiles>
                           <suiteXmlFile>team1.xml</suiteXmlFile>
                           <suiteXmlFile>team2.xml</suiteXmlFile>
                           <suiteXmlFile>team3.xml</suiteXmlFile>
                       </suiteXmlFiles>
   
                       <parallel>tests</parallel>
                       <threadCount>15</threadCount>
                    </configuration>
   
                </plugin>
   `
   
   
![Image](https://github.com/user-attachments/assets/f9eca101-8fed-474a-9381-bd8569efb8fd)
   
   since in the second example suites are run sequentially, there are 5 
sessions free while suites 1 and 2 are running.
   
   
   
   It is possible to run all suites in parallel and have all tests/classes in 
them use a global number/limit of threads?
   
   is it possible to change the number of threads (threadCount) for each suites 
dynamically ?
   
   
   here real example from our production
   
   
![Image](https://github.com/user-attachments/assets/66185ebe-1eb9-4c5d-a055-d126bb31fedb)
   
   number 1 -- a suite with thread-count=1 and minimum number of tests. it 
ended before the others. the free session left after it is idle.
   
   number 2 -- similar.
   
   number 3 -- the set that runs the longest of all sets.
   
   The vertical purple line is the ideal time spent on tests, assuming that the 
entire quota (20 in this case) is used.
   
   
   
   
   code:
   suites:
   https://gist.github.com/VZEROALL/27acc1216f0be7f0e481546f399e4827
   https://gist.github.com/VZEROALL/6ec11b39b2d52227d219bccb7137e801
   https://gist.github.com/VZEROALL/a7332da802abba308d9288a44a46bc42
   
   test class:
   https://gist.github.com/VZEROALL/5dc21dd24fc7b45f282f7a3c35696304
   
   example1 config
   https://gist.github.com/VZEROALL/187118b93ce6464840e4066b4df8c2e3
   example2 config
   https://gist.github.com/VZEROALL/ec33f2f3cd500c9c582f433918f9634a
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@maven.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org