> On Nov 1, 2020, at 10:30 AM, Satish Balay <[email protected]> wrote:
>
> Just a note: subtree workflow is currently broken with petsc4py
>
> We've imported petsc4py using subtree - but now we are unable to export back
> changes in petsc repo to the standalone petsc4py repo due to git subtree
> errors.
>
> I spent some time on workarounds - but didn't go anywhere.
>
> [likely this is a non-issue with this examples repo workflow - as there is no
> direct push back to upstream repo]
>
> And with subtree - not all developers need to know about it. Only the person
> who is syncing the 2 repos.. [if they know how to sync/resolve conflicts]
Since each person is reponsable for their branch getting merged, doesn't this
mean anyone who changes something in PETSc that requires a change in p4pdes has
to fix it via gittree process ( that is have have to sync the 2 repose?) and
hence now how to sync 2 repos via gittree.
Until I see the entire gittree workflow I will not understand.
Barry
>
> Satish
>
> On Sun, 1 Nov 2020, Jed Brown wrote:
>
>> Barry Smith <[email protected]> writes:
>>
>>>>> git subtree does require use of new commands every time you mess with it
>>>>> (say every three months) that we do not know and since each of
>>>>> us will do this infrequently it is likely we will not remember them (I
>>>>> won't) while my approach does not require remembering new commands.
>>>>> My approach only requires branching, push, and pulling on the fork and
>>>>> updating EdsRepo.py which is things all the developers know about and
>>>>> do regularly since we update other external packages.
>>>
>>>>> Circular dependencies (PETSc CI depends on Ed's p4pdes depends on PETSc)
>>>>> are significantly more labor-intensive and it would need to be done on
>>>>> each change, versus once per release cycle.
>>>
>>> I think we are not communicating on exactly the same wave length.
>>>
>>> I am not advocating we test on Ed's I am advocating we test on our fork
>>> of Ed's and how often our fork gets passed to Ed with MR is a choice all of
>>> us make together, now we could do it once at release time with all the
>>> fixes we made, every month or each time. Totally up to Ed how often he
>>> wants to get them, and he is free to ignore them for weeks (up to the next
>>> release if he likes) since our testing will continue because we use our
>>> fork. How often we pass them on to Ed is not related to how we maintain our
>>> CI.
>>>
>>> There is no circular dependency on my approach with Ed's p4pdes.
>>
>> The circular dependency is on our fork of p4pdes.
>>
>>> I vote to fix things in our fork or gittree thing continuously since it
>>> makes it easier to fix things rather than wait to the release when we try
>>> to find and fix everything and it also helps tell us if we introduced a
>>> real bug into PETSc and fix PETSc immediately instead of waiting up to 6
>>> months, just like we now we test immediately with Petsc4py and we should do
>>> with SLEPc. How often we give the updates to Ed is a completely different
>>> issue.
>>>
>>> So again back to my original statement ,it comes down to if the subtree
>>> or the fork approach is easier for all the PETSc developers who do not
>>> currently know gittree and would need to learn it with your approach. I
>>> don't know which is easier learning to use gittree which has its own
>>> gotcha's or using mine which we all know but may require an extra step (not
>>> involving Ed, just updating the p4pdes.py commit each time we change
>>> something in the fork.)
>>>
>>> I think using --download-p4pdes on a couple of systems in the CI is
>>> enough, I don't think we need to put it all CI pipelines (I would like
>>> slepc in all pipelines).but we could put in all pipelines if we want.
>>>
>>> For completeness I show the exact the work flow for my suggestion
>>>
>>> pipelines --download-p4pdes and runs its tests
>>> if it breaks the developer uses --download-p4pdes on their system
>>> they fix the problem either by fixing PETSc or what is downloaded from
>>> the p4pde fork
>>> if the fix is in the p4pdes fork they make a branch in the p4pdes
>>> fork, which they already have since they used --download-p4pdes and thus
>>> have the fork on their system
>>> they put the fix in new branch in the p4pdes fork and push it
>>> they edit p4pdes.py and put a new commit in it pointing to their
>>> branch in the fork
>>> run the pipeline again
>>> if fails with p4pdes they do the above again
>>> else the PETSc branch gets accepted and merged to master
>>> depending on Ed's choice we make an MR for p4pdes depending on the
>>> agreed upon cycle. If Ed puts the fix into his master then we just update
>>> our fork with his latest master with a simple merge of his.
>>> This will then become the new one we test against. If Ed doesn't respond to
>>> the MR all is fine we
>>> just continue on our fork. If he puts other things in his
>>> branch but not our MR we just merge that into our fork and so are still
>>> testing with his latest master.
>>
>> As you've enumerated here, this requires three MRs per change:
>>
>> 1. in PETSc with the actual change and to point at a p4pdes commit
>> 2. in p4pdes (Ed's or our fork) to implement needed changes (note: this
>> can't merge until #1 merges)
>> 3. in PETSc to point at the merge commit of p4pdes (after #2 merges)
>>
>> With subtree, there is only one MR and it's in the PETSc repository.
>>
>> Recall that we exchanged hundreds of emails to subtree in BuildSystem long
>> ago, then quite a lot more to subtree in petsc4py recently, and now we're
>> doing it again.
>>
>> The arguments against subtreeing p4pdes are
>>
>> 1. We want an extra hurdle so developers think twice before changing those
>> interfaces
>> 2. We want to drive traffic to Ed's repo
>> 3. The repository is so big we don't want every `git clone
>> gitlab:petsc/petsc` to include it
>>
>> This repo is small so I'm most sympathetic to #2, but nobody has made these
>> arguments yet.
>>
>>> This happens to be the exact same thing I do now with slepc and any other
>>> git based package that I use now (except for some I need to make my own
>>> fork of the external package because we don't have one always available
>>> eventually likely we will likely put more forks into gitlab/petsc so each
>>> person doesn't constantly need to make fresh forks of external packages)
>>>
>>> If subtree requires fewer steps than this and has no new oddball git
>>> subtree commands then we should definitely use subtree, if it requires
>>> other steps involving subtree we need see those commands written in a
>>> workflow like I have written above and decide if it is still simpler or not.
>>>
>>> I am not rejecting subtree, we just need to explicitly see its complete
>>> work flow and hence the differences to decide, that is what I asking for.
>>>
>>> Barry
>>>
>>> It seems to me that with subtree we also need to maintain a fork of Ed's
>>> stuff or else that will have a circular dependency. But perhaps I do not
>>> understand it.
>>
>> Such a fork would only be used once per release to submit PRs to Ed.
>>
>
