OK, I already found some places where "/tmp" is hard-coded: https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L717-L719 https://github.com/apache/cassandra/blob/trunk/test/unit/org/apache/cassandra/db/DirectoriesTest.java#L757-L759
Can I open a ticket to track fixes for these and any other issues I run into while moving to using "build/tmp"? Thanks, Derek On Sun, Aug 13, 2023 at 10:13 AM Josh McKenzie <jmcken...@apache.org> wrote: > I think we want/need relative paths, e.g. "build/tmp", and if the path is > in a mounted volume there can be another container still running. > > Sure. The specifics of *what* path isn't interesting to me. > > The pattern of: > 1. Let env declare where TEMP lives > 2. Write things to TEMP > 3. Delete things from TEMP every time we run a new suite or do "ant clean" > > Is. > > Could also take it a step further and let env declare RESULTS_PATH for > things they want to be durable and add an "ant clean-results" target. > > On Sun, Aug 13, 2023, at 11:33 AM, Derek Chen-Becker wrote: > > Nevermind, I found "tmp.dir" > > On Sun, Aug 13, 2023 at 9:29 AM Derek Chen-Becker <de...@chen-becker.org> > wrote: > > Cool, > > I'm a little confused. Is "tmp.dir" a custom Java property that we expose? > I thought that the standard "property was "java.io.tmpdir". Let me take a > stab at setting tmp.dir to build/tmp and see if I run into any issues (or > still see any files in /tmp). > > Cheers, > > Derek > > On Sun, Aug 13, 2023 at 8:24 AM Mick Semb Wever <m...@apache.org> wrote: > > > While doing some local testing, I noticed that my /tmp drive completely > filled with test artifact files (e.g. data directories, logs, commit logs, > etc). Mick pointed out that we do attempt to do some "find" based cleanup > in CI ( > https://github.com/apache/cassandra-builds/blob/trunk/jenkins-dsl/cassandra_job_dsl_seed.groovy#L437-439), > but I was wondering if it might be better to do the following for direct > ant builds: > > 1. If TMPDIR is set, use it. It does not appear to be honored, currently, > so I need to do some analysis of what would need to be done here > 2. If TMPDIR is not set, use "mktemp" to create a temp directory and set > TMPDIR with that directory > 3. Update the "ant clean" task to delete TMPDIR when we've generated it, > or attempt the find-based cleanup if TMPDIR was provided > > Does anyone know if there are any hard-coded assumptions that test files > will live directly under /tmp? > > > > This will need testing with in-tree scripts, ci-cassandra, and circleci :( > > What comes to mind: > - TMPDIR works best today with the python and scripting stuff > - setting TMPDIR can break tests, hence unit test script set instead > $TMP_DIR which is passed to `-Dtmp.dir=…` > - /tmp is often set up to be a more appropropriate fs (and volume size) > - it is hard to customise everything > - it needs to work locally on your machine as well as in docker > containers, as well as CI > > If we want something that is wiped by `ant clean` I would suggest using > the build/tmp directory by default. > In-tree scripts do this for unit tests: > https://github.com/apache/cassandra/blob/trunk/.build/run-tests.sh#L160 > but are not yet doing it for the dtests: > https://github.com/apache/cassandra/blob/trunk/.build/run-python-dtests.sh#L58 > > > So I don't think we need (3). If the caller has specified TMPDIR it is > then their responsibility to clean it. > > We can probably avoid trying to set TMPDIR, instead defaulting the > `tmp.dir` property to the build/tmp directory. > > The goal of any changes in build.xml should be, in addition to providing > the best dev exp, to simplify the testing and CI layers above it. > > > > -- > +---------------------------------------------------------------+ > | Derek Chen-Becker | > | GPG Key available at https://keybase.io/dchenbecker and | > | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org | > | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC | > +---------------------------------------------------------------+ > > > > -- > +---------------------------------------------------------------+ > | Derek Chen-Becker | > | GPG Key available at https://keybase.io/dchenbecker and | > | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org | > | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC | > +---------------------------------------------------------------+ > > > -- +---------------------------------------------------------------+ | Derek Chen-Becker | | GPG Key available at https://keybase.io/dchenbecker and | | https://pgp.mit.edu/pks/lookup?search=derek%40chen-becker.org | | Fngrprnt: EB8A 6480 F0A3 C8EB C1E7 7F42 AFC5 AFEE 96E4 6ACC | +---------------------------------------------------------------+
