replies are inline to your inline replies to my inline replies đ„
> We can ask INFRA to set up a separate snapshots repository just for us, > with a longer expiry policy. I'd rather not create extra work for infra if > there's other ways we can do this, and this approach would always require > some fallback approach to rebuilding the dedepency's SHA from scratch. > > > If they will allow this and allow the snapshots to never be purged, then I > am ok with this as a solution. > They will get purged eventually, and may get lost (no backups). > > >> We break python-dtest when cross-cutting changes are added as CI is hard >> to do correctly or not supported (testing downstream users (our 4 supported >> branches) is rarely done). >> > > > python dtests' is also in a different category, (the context and > consumption in a different direction, i.e. it's not a library used within > the in-tree). > > > I disagree. The point I was making is we have several dependencies and we > should think about how we maintain them. My point is still valid that > python dtests are involved with cross cutting changes to Cassandra, and the > lack of downstream testing has broken us several times. The solution to > this problem may be different than Accord (as C* doesnât depend on python > dtest as you point out), but that does not mean we shouldnât think about it > in this conversationâŠ. > > One thing that comes to mind is that dependencies may benefit from running > a limited C* CI as part of their merge process. At the moment people are > expected to create a tmp CI branch for all 4 supported C* versions, point > it to the python dtest change, then submit to the JIRA as proof that CI was > ran⊠normally when I find python dtest broke in branch X I find this had > not happened⊠> > This holds true I believe for JVM dtest as well as we should be validating > that the 4 target C* branches still work if you are touching jvm dtest⊠> > Now, with all that, Accord being external will have similar issues, a > change there may break Cassandra so we should include a subset of Cassandra > tests in Accordâs CI. > Fair enough, and this reasoning also applies to dtest-api. But this is an additional concern in the discussion, with potentially different solutions. Part of the testing requirements to dtests (and libraries that are included in-tree) is downstream CI. When you make a change in cassandra-dtest, you shouldn't have to go test the C* branches â it should be part of the CI pipeline for cassandra-dtest itself. For dtests the versions tested are explicit. It's different for libraries that are included in-tree, but you have to make the change in-tree, so it makes sense it's part of in-tree CI. > * [nice to have] be able to work with all subprojects in one IDE and not >> have to switch between windows while making cross-cutting changes >> > Isn't it only IntelliJ that suffers this problem? (That doesn't invalidate > it, just askingâŠ) > > I have not used Eclipse or NetBeans for around 10 years so no clue! > > > >> but this can be confusing for people who mainly build in IDE and donât >> depend on build.xml until later in development⊠this is something we should >> think about⊠>> > Again, isn't this only IntelliJ? > > Not sure, the only other IDE we support is NetBeans and not sure what we > do there. > Off-topic: NetBeans allows you to have many projects open in the one window (easy to have 20-30 projects open), and it does not do anything with sources its own way â everything is delegated to the project's build system (ant/gradle/maven). A project I am familiar with has their build auto-inject git hooks to make >> sure things âjust workâ, we may be able to solve this in a similar way? >> > > I'd like to hear/see more! > > The project wants to make sure commit messages are structured âcorrectlyâ > so enforces this via git hooks. Gradle (the build they use) makes tasks > depend on âinstallGitHooksâ which copies 2 hooks to .git/hooks (commit-msg, > and pre-push) > > We could always do the same in build.xml, that copies hooks we define into > .git/hooks to make sure the behaviors we expect are enforced. See > https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks > <https://urldefense.com/v3/__https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks__;!!PbtH5S7Ebw!c8I827DalUUwWD3xz9aSE9hc0l1_GORe4z66HZlMTELcLJLR3h79Ybgl-ETe3MEQl43j3acm60mczw$> > > There is a âpost-checkoutâ hook we could leverage to detect that the > dependences SHA are no longer the same and recursively checks out the > âcorrectâ dependencies > I like it! But I think we would need this AND a fail-fast in the build.xml > - our releases get more complicated (our source tarballs are the asf >> releases) >> >> >> We donât include our dependencies do we? If so, then does it really? If >> Accord is a library we use, why would we include itâs source in the build? >> Isnât it just another library from this point of view? >> > > The build of the source tarball must work. If the source tarball release > switches how it does things, from building the submodule to including a > dependency then we're back to having to make releases (and introducing a > risk, and we don't ourselves work frequently with the source tarballs). > > Thatâs fair. We wouldnât have a .git dir anymore so couldnât rely on git > to âdo the right thingâ in this context, so bundling the dependencies may > be the simplest in the git submodules case. > The source tarball is built here: https://github.com/apache/cassandra/blob/trunk/build.xml#L892-L937 Both a git post-checkout and a build fail-fast will protect us here. But the post-checkout will need to fail silently if the .git subdirectory doesn't exist. > Its been a long time since I used git submodules so before commenting I > created a project to test this out and get a better feel for it. > > $ cat .gitmodules > [submodule "apache-cassandra"] > path = apache-cassandra > url = https://github.com/apache/cassandra.git > $ cat .git/modules/apache-cassandra/HEAD > b07c3127cce7e1a8b9d4c34e1359539e12f1b4a7 > > The git repo contains the SHA, which is stable until someone commits a > change to it. If the submodule has external updates, that doesnât get > pulled in > If you roll back to an older commit, need to update the submodules > > git submodule update > I'm presuming the post-checkout hook automates this for us? (It would be nice there's not the additional step required every time you switch/checkout/bisectâŠ)
