> I think we want/need relative paths, e.g. "build/tmp", and if the path is in > a mounted volume there can be another container still running. Sure. The specifics of *what* path isn't interesting to me.
The pattern of: 1. Let env declare where TEMP lives 2. Write things to TEMP 3. Delete things from TEMP every time we run a new suite or do "ant clean" Is. Could also take it a step further and let env declare RESULTS_PATH for things they want to be durable and add an "ant clean-results" target. On Sun, Aug 13, 2023, at 11:33 AM, Derek Chen-Becker wrote: > Nevermind, I found "tmp.dir" > > On Sun, Aug 13, 2023 at 9:29 AM Derek Chen-Becker <de...@chen-becker.org> > wrote: >> Cool, >> >> I'm a little confused. Is "tmp.dir" a custom Java property that we expose? I >> thought that the standard "property was "java.io.tmpdir". Let me take a stab >> at setting tmp.dir to build/tmp and see if I run into any issues (or still >> see any files in /tmp). >> >> Cheers, >> >> Derek >> >> On Sun, Aug 13, 2023 at 8:24 AM Mick Semb Wever <m...@apache.org> wrote: >>> >>>> While doing some local testing, I noticed that my /tmp drive completely >>>> filled with test artifact files (e.g. data directories, logs, commit logs, >>>> etc). Mick pointed out that we do attempt to do some "find" based cleanup >>>> in CI >>>> (https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439), >>>> but I was wondering if it might be better to do the following for direct >>>> ant builds: >>>> >>>> 1. If TMPDIR is set, use it. It does not appear to be honored, currently, >>>> so I need to do some analysis of what would need to be done here >>>> 2. If TMPDIR is not set, use "mktemp" to create a temp directory and set >>>> TMPDIR with that directory >>>> 3. Update the "ant clean" task to delete TMPDIR when we've generated it, >>>> or attempt the find-based cleanup if TMPDIR was provided >>>> >>>> Does anyone know if there are any hard-coded assumptions that test files >>>> will live directly under /tmp? >>> >>> >>> This will need testing with in-tree scripts, ci-cassandra, and circleci :( >>> >>> What comes to mind: >>> - TMPDIR works best today with the python and scripting stuff >>> - setting TMPDIR can break tests, hence unit test script set instead >>> $TMP_DIR which is passed to `-Dtmp.dir=…` >>> - /tmp is often set up to be a more appropropriate fs (and volume size) >>> - it is hard to customise everything >>> - it needs to work locally on your machine as well as in docker >>> containers, as well as CI >>> >>> If we want something that is wiped by `ant clean` I would suggest using the >>> build/tmp directory by default. >>> In-tree scripts do this for unit tests: >>> https://github.com/apache/cassandra/blob/trunk/.build/run-tests.sh#L160 >>> but are not yet doing it for the dtests: >>> https://github.com/apache/cassandra/blob/trunk/.build/run-python-dtests.sh#L58 >>> >>> >>> So I don't think we need (3). If the caller has specified TMPDIR it is then >>> their responsibility to clean it. >>> >>> We can probably avoid trying to set TMPDIR, instead defaulting the >>> `tmp.dir` property to the build/tmp directory. >>> >>> The goal of any changes in build.xml should be, in addition to providing >>> the best dev exp, to simplify the testing and CI layers above it. >>> >> >> >> -- >> +---------------------------------------------------------------+ >> | Derek Chen-Becker | >> | GPG Key available at https://keybase.io/dchenbecker and | >> | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org | >> | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC | >> +---------------------------------------------------------------+ >> > > > -- > +---------------------------------------------------------------+ > | Derek Chen-Becker | > | GPG Key available at https://keybase.io/dchenbecker and | > | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org | > | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC | > +---------------------------------------------------------------+ >
