[
https://issues.apache.org/jira/browse/CAMEL-23795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aurélien Pupier updated CAMEL-23795:
------------------------------------
Description:
it finishes by filling up the Disk space of Jenkins Nodes.
Observation so far:
* several old workspaces had test-infra/camel-test-infra-cli/target/data folder
with root owner ; this part has been modified so that there is none of them
left after the build. And anyways, seems that it was the case with 4.18.x and a
bunch of teh repos were correctly cleaned up
* the files in test-infra/camel-test-infra-cli/target/tmp-repo has linux1 owner
and I cannot delete them with a Jenkins freestyle project
{noformat}
+ ls -l
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1
total 20
-rw-r--r-- 1 linux1 linux1 2497 Oct 13 2009 javax.inject-1.jar
-rw-r--r-- 1 linux1 linux1 40 Jan 30 2025 javax.inject-1.jar.sha1
-rw-r--r-- 1 linux1 linux1 612 Oct 13 2009 javax.inject-1.pom
-rw-r--r-- 1 linux1 linux1 40 Jan 30 2025 javax.inject-1.pom.sha1
-rw-r--r-- 1 linux1 linux1 196 Jan 30 2025 _remote.repositories
{noformat}
{noformat}
rm -R
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738284235794
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738314457199
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738331577601
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738697517079
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738943273734
rm: cannot remove
'/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/_remote.repositories':
Permission denied
rm: cannot remove
'/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/javax.inject-1.jar.sha1':
Permission denied
rm: cannot remove
'/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/javax.inject-1.pom.sha1':
Permission denied
{noformat}
Each workspace seems to take around 4G. The total size of nodes are 900G, so we
are filling them with around 200 builds on the exact same node, but there are
several nodes available
created this job for investigation
https://ci-builds.apache.org/job/Camel/job/Debug%20attempt%20of%20ws-cleanup%20from%20freestyle%20project
was:
it finishes by filling up the Disk space of Jenkins Nodes.
Observation so far:
* several old workspaces had test-infra/camel-test-infra-cli/target/data folder
with root owner ; this part has been modified so that there is none of them
left after the build. And anyways, seems that it was the case with 4.18.x and a
bunch of teh repos were correctly cleaned up
* the files in test-infra/camel-test-infra-cli/target/tmp-repo has linux1 owner
and I cannot delete them with a Jenkins freestyle project
{noformat}
+ ls -l
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1
total 20
-rw-r--r-- 1 linux1 linux1 2497 Oct 13 2009 javax.inject-1.jar
-rw-r--r-- 1 linux1 linux1 40 Jan 30 2025 javax.inject-1.jar.sha1
-rw-r--r-- 1 linux1 linux1 612 Oct 13 2009 javax.inject-1.pom
-rw-r--r-- 1 linux1 linux1 40 Jan 30 2025 javax.inject-1.pom.sha1
-rw-r--r-- 1 linux1 linux1 196 Jan 30 2025 _remote.repositories
{noformat}
{noformat}
rm -R
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738284235794
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738314457199
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738331577601
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738697517079
/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738943273734
rm: cannot remove
'/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/_remote.repositories':
Permission denied
rm: cannot remove
'/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/javax.inject-1.jar.sha1':
Permission denied
rm: cannot remove
'/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/javax.inject-1.pom.sha1':
Permission denied
{noformat}
Each workspace seems to take around 4G. The total size of nodes are 900G, so we
are filling them with around 200 builds on the exact same node, but there are
several nodes available
> Camel jobs are not cleaned up successfully on Jenkins CI
> --------------------------------------------------------
>
> Key: CAMEL-23795
> URL: https://issues.apache.org/jira/browse/CAMEL-23795
> Project: Camel
> Issue Type: Task
> Components: build system
> Affects Versions: 4.20.0
> Reporter: Aurélien Pupier
> Assignee: Aurélien Pupier
> Priority: Major
>
> it finishes by filling up the Disk space of Jenkins Nodes.
> Observation so far:
> * several old workspaces had test-infra/camel-test-infra-cli/target/data
> folder with root owner ; this part has been modified so that there is none of
> them left after the build. And anyways, seems that it was the case with
> 4.18.x and a bunch of teh repos were correctly cleaned up
> * the files in test-infra/camel-test-infra-cli/target/tmp-repo has linux1
> owner and I cannot delete them with a Jenkins freestyle project
> {noformat}
> + ls -l
> /home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1
> total 20
> -rw-r--r-- 1 linux1 linux1 2497 Oct 13 2009 javax.inject-1.jar
> -rw-r--r-- 1 linux1 linux1 40 Jan 30 2025 javax.inject-1.jar.sha1
> -rw-r--r-- 1 linux1 linux1 612 Oct 13 2009 javax.inject-1.pom
> -rw-r--r-- 1 linux1 linux1 40 Jan 30 2025 javax.inject-1.pom.sha1
> -rw-r--r-- 1 linux1 linux1 196 Jan 30 2025 _remote.repositories
> {noformat}
> {noformat}
> rm -R
> /home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190
>
> /home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738284235794
>
> /home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738314457199
>
> /home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738331577601
>
> /home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738697517079
>
> /home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738943273734
> rm: cannot remove
> '/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/_remote.repositories':
> Permission denied
> rm: cannot remove
> '/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/javax.inject-1.jar.sha1':
> Permission denied
> rm: cannot remove
> '/home/jenkins/jenkins-home/712657a4/workspace/l_Camel_Core_Build_and_test_main_ws-cleanup_1738256760190/test-infra/camel-test-infra-cli/target/tmp-repo/javax/inject/javax.inject/1/javax.inject-1.pom.sha1':
> Permission denied
> {noformat}
> Each workspace seems to take around 4G. The total size of nodes are 900G, so
> we are filling them with around 200 builds on the exact same node, but there
> are several nodes available
> created this job for investigation
> https://ci-builds.apache.org/job/Camel/job/Debug%20attempt%20of%20ws-cleanup%20from%20freestyle%20project
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
