> On 15 Jun 2018, at 10:29, Tim Mackinnon <tim@testit.works> wrote: > > > In many parts of the dev world - every commit should be shippable, meaning > atomic - particularly with the use of CI systems (Travis, Gitlab etc) that > build on every commit.
but then, you mean every *push* should be shippable :) that’s something I agree :) > > Of course you can choose to work a different way - but our tools shouldn’t > force a very common practice to be awkward, should they? > > I don’t understand why many of you guys are so anti such a common thing. I’m not anti. Being “anti” requires a lot of effort I prefer to spend on something else :) But in general, I prefer many small commits of tasks. And yes, I prefer those commits to be atomic, but when is not possible I just make sure I do not push until something is finished. > In fact our community pioneered this way of working - a build machine where > you loaded all your code and ran all your tests before committing. Also, a > large number of agile teams will council to work on master (and avoid > branches - a technique we also pioneered). hah… we pioneered it as a technique basically because our tools were not capable of branching properly, but well, It can be a way to minimise integration problems, yes (depends on the team, of course). > In today’s world where we live in a wider polyglot stack this is one of the > reasons why git is so attractive as it should easily embraces that. > > I get that it’s not quite possible yet - but there are some decent ways to > accommodate this (a staging operation, or perhaps a better way to make use of > the #addToIndex: which I will experiment with). I think the easiest/fastest way to handle this is to add a “special commit” plugin which could export sources and then delegate commit to an external tool. And it could be a plugin because then people wanting to work with it can load it independently. > Anyway, I’ll shut up - and possibly will learn how to implement it myself. > Thinking more, I may also be able run with commit and no push , which I will > try - as it’s probably the easiest mechanism if the CI will recognise that (I > think it will). do not shut up :) your opinions and considerations are good and a good way to see a different pov that may be better than ours. > > Thanks for the enlightening conversation, and as always the hard work on > iceberg that made this conversation even possible. thank you Esteban > > Tim > > Sent from my iPhone > > On 15 Jun 2018, at 07:11, Norbert Hartl <norb...@hartl.name > <mailto:norb...@hartl.name>> wrote: > >> >> >> Am 14.06.2018 um 13:12 schrieb Thierry Goubier <thierry.goub...@gmail.com >> <mailto:thierry.goub...@gmail.com>>: >> >>> Hi Norbert, Tim, >>> >>> 2018-06-14 11:33 GMT+02:00 Norbert Hartl <norb...@hartl.name >>> <mailto:norb...@hartl.name>>: >>> >>> >>>> Am 14.06.2018 um 10:30 schrieb Tim Mackinnon <tim@testit.works >>>> <mailto:tim@testit.works>>: >>>> >>>> Hi - yes I’m pleased you check out the entire tree, although currently >>>> it’s a bit confusing that you do (fortunately this does give the >>>> possibility that we can checkout images and other resources that an Pharo >>>> application might rely on - without having to resort to the Seaside >>>> FileLibrary trick). >>>> >>>> However my concrete case was that I have a gitlab ci pipeline and next to >>>> my src directory in my project I have a config directory that has some >>>> Nginx config for my teapot app. If I add a teapot route, I also need to >>>> adjust that config and check both changes in together. I can’t easily do >>>> that now? >>>> >>>> I can modify /config/app.nginx either in another app (intellij) or even in >>>> the simple Pharo text editor, and the I can add my new route in my >>>> DemoApp>>createRoutes method but how do I check them in together so my >>>> pipeline will build atomically? >>>> >>>> Iceberg hasn’t written out the changes yet, so IntelliJ can’t see them to >>>> do a commit, and iceberg ignores the parallel /config directory (that it >>>> checked out). So it’s a catch 22. >>>> >>>> This is why I suggested maybe we could specify safer (textual) directories >>>> that iceberg might also checkin? OR we have a Stage command in iceberg >>>> that does everything that commit does up to the point of actually writing >>>> to the repo - then I could jump to IntelliJ and do the final commit there >>>> and use its tools to manage non Pharo stuff (until we can build more)? >>>> >>>> Does this make sense? >>>> >>> I don’t understand why you are so eager to have everything into one commit. >>> Usually the tension is rather have small commits. What is the problem of >>> having two commits for this? >>> >>> A single commit allow one to make sure that both the smalltalk code and the >>> external resource are well in sync, and that you may not ending up in a >>> situation were at commit j has data format v1 and smalltalk code for format >>> v2, because it is in commit j+1 that you rewrote the data in format v2. >>> >> Yes, sure but that is only a problem if you lost control over which commit >> goes into action. I think the problem is in the deployment then. Putting >> integrity constraints on commits is IMHO the wrong level deployment >> granularity >> >> Norbert >> >>> I had that issue recently with a Pharo / FPGA project with a Pharo package >>> generating state machines to be integrated in a Verilog FPGA design with C >>> code for the drivers, the softcore in the FPGA, and test programs, and both >>> Pharo and the C/FPGA working out of the same test data... And that whole >>> thing getting regularly out of sync. >>> >>> IMHO: I'd model a concept of "multi-lingual projet" for that sort of >>> things, where one can list, in Pharo, Monticello packages and external >>> files or directories. Whatever the technology you use (OSProcess, libgit, >>> whatever...) it is easy then to commit all this in a single pass. On a >>> per-package basis, I'd see the package manifest as a suitable place for >>> listing external resources. On a per-project basis, a possible place could >>> be the manifest for the BaselineOf the project. >>> >>> Thierry >>> >>> >>> Norbert >>> >>>> As an aside - I’d really like to checkin in the play-xxx directories (the >>>> .ph files) as there is often useful playground stuff I’d like to access on >>>> my home computer. We can’t do that easily at the moment either. >>>> >>>> Tim >>>> >>>> Sent from my iPhone >>>> >>>> On 14 Jun 2018, at 09:12, Guillermo Polito <guillermopol...@gmail.com >>>> <mailto:guillermopol...@gmail.com>> wrote: >>>> >>>>> Just to complement Esteban's answer: >>>>> >>>>> - Iceberg checks out in disk more than the src directory because you >>>>> **may** want to edit files from the command line, and after long >>>>> discussions we did not want to forbid that. >>>>> Actually, just to put everybody in perspective, at first the idea was to >>>>> not have a working copy in disk at all, but just hit to the blob. >>>>> Imagine is nowadays we are a bit alien, that would have been worst :) >>>>> >>>>> - About checking in files. I'd like to understand what you mean exactly. >>>>> - Do you want to load them into memory? >>>>> This would be the "more consistent" way to do it, following the "the >>>>> image it its own working copy" metaphore. >>>>> This would allow us to, for example, share an image and transparently >>>>> share resources with it (without requiring to clone). >>>>> But this would have some impact in memory consumption and add stress >>>>> to the GC, right? >>>>> >>>>> - Or do you mean to ask like any other Git client and show you the file >>>>> differences between the working copy and the git index? >>>>> The problem with this approach is that we will have some treatment >>>>> for pharo code and some different treatment for non-code... >>>>> If I do a change to a class, the change is kept in the image. But if >>>>> I do a change to a file, that change is not kept in the image! >>>>> >>>>> Also, as Esteban says, having an IDE with support for files would mean >>>>> that we would need good tools to edit in-memory files (not only text >>>>> files, right? but also any kind of binary file...) >>>>> >>>>> So far we cover the bare minimum that allows us to *not lose* changes :) >>>>> >>>>> On Wed, Jun 13, 2018 at 11:07 PM Tim Mackinnon <tim@testit.works >>>>> <mailto:tim@testit.works>> wrote: >>>>> >>>>> > yeah… but is a lot of work and no time just right now. >>>>> > long term, it would be cool to manage everything from iceberg. >>>>> > but reality check, is a huge amount of work so it has to come step by >>>>> > step. >>>>> >>>>> Fair enough - its pretty cool we’ve got this far, and I guess the onus is >>>>> on the rest of us to learn more about how its done and see if we can >>>>> contribute more somehow. I really appreciate the love you’ve already put >>>>> into this - it works far better than I think we even realised it could. >>>>> >>>>> Tim >>>>> >>>>> >>>>> >>>>> > On 13 Jun 2018, at 21:55, Esteban Lorenzano <esteba...@gmail.com >>>>> > <mailto:esteba...@gmail.com>> wrote: >>>>> > >>>>> > >>>>> > >>>>> >> On 13 Jun 2018, at 22:44, Tim Mackinnon <tim@testit.works >>>>> >> <mailto:tim@testit.works>> wrote: >>>>> >> >>>>> >> Esteban - so I don't then understand why iceberg (usefully in my view) >>>>> >> checks out more than the src directory, if it’s only focusing on the >>>>> >> Pharo blob? >>>>> >> >>>>> >> I’m guessing that by knowing where the src is, you are just committing >>>>> >> that part of the tree with libgit? >>>>> >> >>>>> >> Perhaps from a pragmatic first step you might consider letting us add >>>>> >> a second safe resources directory that you could check in atomically >>>>> >> as well (on the understanding all bets are off if it goes wrong?) >>>>> >> >>>>> >> OR could we have a check in mode does all the add/remove operations >>>>> >> and writes to disk but then let’s you drop to the command line/other >>>>> >> tool to add any other files and do the final commit? >>>>> >> >>>>> >> I just feel like you/we are so close to something that works a bit >>>>> >> more broadly and embrace the wider world.? >>>>> > >>>>> > yeah… but is a lot of work and no time just right now. >>>>> > long term, it would be cool to manage everything from iceberg. >>>>> > but reality check, is a huge amount of work so it has to come step by >>>>> > step. >>>>> > >>>>> >> >>>>> >> Tim >>>>> >> >>>>> >> Sent from my iPhone >>>>> >> >>>>> >>> On 13 Jun 2018, at 21:28, Esteban Lorenzano <esteba...@gmail.com >>>>> >>> <mailto:esteba...@gmail.com>> wrote: >>>>> >>> >>>>> >>> hi, >>>>> >>> >>>>> >>> >>>>> >>>> On 13 Jun 2018, at 16:50, Tim Mackinnon <tim@testit.works >>>>> >>>> <mailto:tim@testit.works>> wrote: >>>>> >>>> >>>>> >>>> Hi - my second attempt at using Pharo with Git has proven very >>>>> >>>> satisfying (I saw the potential in phase 1, but it was often >>>>> >>>> difficult to understand what was happening and the workflow to use). >>>>> >>>> >>>>> >>>> One thing that has come up a few times for me however - and its >>>>> >>>> something that using git nicely highlights, there are many >>>>> >>>> non-smalltalk assets in my project that don’t need to live in the >>>>> >>>> image (like Seaside FileLibraries were trying to do) but do need to >>>>> >>>> be versioned and be part of my project. Common examples are server >>>>> >>>> config files, images and even the playground history files that are >>>>> >>>> useful to pull up when on another computer. >>>>> >>>> >>>>> >>>> It seems that while Iceberg does check out a full project, if I >>>>> >>>> change any of the files outside of the src directory (like edit a >>>>> >>>> .txt file using the crude Pharo file editor), those changes don’t >>>>> >>>> get committed when I do a checkin? Is this on purpose? It makes the >>>>> >>>> workflow a bit trickier to do an atomic commit of a piece of work - >>>>> >>>> and I’m not clear whether this is a conscious thing, or an MVP thing >>>>> >>>> (and it will come later). >>>>> >>> >>>>> >>> workflow is tricker because you are expecting iceberg to talk with >>>>> >>> the local working copy and to handle that WC. >>>>> >>> what happens in fact is different: iceberg treats the image as a >>>>> >>> working copy itself (it has its own “stage” area) and what you have >>>>> >>> in disk is like a separated WC. >>>>> >>> >>>>> >>> at least, this is the metaphor we are using now, because we cannot >>>>> >>> realistically handle/control what is in disk since it can be >>>>> >>> anything. >>>>> >>> So, instead having this picture in mind: >>>>> >>> >>>>> >>> Image -> Disk -> Git blob (database) >>>>> >>> >>>>> >>> you need to have this other: >>>>> >>> >>>>> >>> Image \ >>>>> >>> Git blob (database) >>>>> >>> Disk / >>>>> >>> >>>>> >>> >>>>> >>> you will see as soon as you change the mental image, your problems >>>>> >>> are gone ;) >>>>> >>> >>>>> >>> cheers! >>>>> >>> Esteban >>>>> >>> >>>>> >>> ps: diagram before is not exactly as it is since the image actually >>>>> >>> writes into disk first, but this is an implementation detail we would >>>>> >>> like to remove in the future, even. >>>>> >>> >>>>> >>>> >>>>> >>>> As mentioned above, I was also thinking it would be nice if I could >>>>> >>>> checkin some of the play-xxxx/*.sh files to essentially keep some of >>>>> >>>> that history synced between environments (or team members?). >>>>> >>>> >>>>> >>>> It strikes me that this is the kind of thing that git integration >>>>> >>>> should bring to us? >>>>> >>>> >>>>> >>>> I can overlay my copy of IntelliJ on top of my local iceberg >>>>> >>>> directory and then use it for checkins - but then I still have the >>>>> >>>> atomic problem, as its only when I commit that tonel files are >>>>> >>>> written out onto the file system for me to checkin along with any >>>>> >>>> other assets I’ve changed. Does anyone else have a good workflow for >>>>> >>>> this? What do you guys do? >>>>> >>>> >>>>> >>>> Tim >>>>> >>> >>>>> >>> >>>>> >> >>>>> >> >>>>> > >>>>> > >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> >>>>> Guille Polito >>>>> Research Engineer >>>>> >>>>> Centre de Recherche en Informatique, Signal et Automatique de Lille >>>>> CRIStAL - UMR 9189 >>>>> French National Center for Scientific Research - http://www.cnrs.fr >>>>> <http://www.cnrs.fr/> >>>>> >>>>> Web: http://guillep.github.io <http://guillep.github.io/> >>>>> Phone: +33 06 52 70 66 13 >>> >>>
