On 19 October 2013 19:20, Kay Schenk <kay.sch...@gmail.com> wrote: > On Fri, Oct 18, 2013 at 7:52 AM, Andre Fischer <awf....@gmail.com> wrote: > > > On 18.10.2013 15:58, janI wrote: > > > >> On 18 October 2013 15:00, Andre Fischer <awf....@gmail.com> wrote: > >> > >> On 18.10.2013 14:02, janI wrote: > >>> > >>> sd > >>>> > >>>> > >>>> On 18 October 2013 13:36, Andre Fischer <awf....@gmail.com> wrote: > >>>> > >>>> On 18.10.2013 11:32, janI wrote: > >>>> > >>>>> Hi. > >>>>> > >>>>>> due to the discussion in thread "Mentor a new build system", I have > >>>>>> made a > >>>>>> proposal for a central Makefile located in main. > >>>>>> > >>>>>> Hi Jan, > >>>>>> > >>>>> it is great that you are going to improve this part of the build > >>>>> system. > >>>>> But I think that we need more details about how the proposed build > >>>>> system > >>>>> works. Without them I can not really evaluate the proposal. > >>>>> > >>>>> First of all, I agree with juergens remarks that this should be > >>>>> > >>>> discussed > >>>> before implemented, hence the wiki page. > >>>> > >>>> Secondly this has nothing directly to do with the proposed build > system, > >>>> its a simple replacement of build.pl in the current system. > >>>> > >>>> Yes, that is how I understood it. I just did not know how to call > the > >>> build.pl replacement. > >>> > >>> > >>> > >>> I know that build.pl works, but having a Makefile in main, would make > >>>> us > >>>> one step closer on being compatible with the distros. To me this job > is > >>>> a > >>>> simple cleanup, not something we deadly need, but nice to have. > >>>> > >>>> > >>>> Some remarks regarding the missing options: > >>>> > >>>>> --from <module> > >>>>> This is one of the more important options and one that I use > >>>>> frequently > >>>>> (also in the form --all:<module>). > >>>>> Note that if you are in <moduleA> and call 'make --from > <moduleB>' > >>>>> then > >>>>> all modules are built > >>>>> a) which <moduleA> depends on > >>>>> b) but not those that <moduleB> depends on > >>>>> c) Both <moduleA> and <moduleB> are built. > >>>>> > >>>>> I have changed the documentation. > >>>>> > >>>> I use the --all:<module> myself very often, and have changed the > >>>> documentation, because it is of course supported. > >>>> > >>>> The difference is that you do the call in main, but that is a minor > >>>> detail > >>>> that can be easily corrected (have <module>/Makefile calling > >>>> main/Makefile. > >>>> > >>>> I have also changed documentation on --html due to juergens comments. > >>>> > >>>> I am not sure that we understand --from and --since in the same way > so > >>> I > >>> will try to explain what I think they do. > >>> > >>> Let's imagine that we have a simple project with modules A, B, C, D and > >>> E. > >>> where B depends on A, C on B, D on C, and E on D. > >>> A ' make all' would mean 'make E'. The dependencies would then lead to > >>> building modules A, B, C, D, E in this order. > >>> If I am in E and call 'make --from C' then only C, D, and E should be > >>> built. A 'make --since C' would only build D and E. > >>> > >>> If I am in D and call 'make --from B' then modules B, C, and D are > built. > >>> Call 'make --since B' to build only C and D. > >>> Note that 'make --from' accepts more than one module name (while 'make > >>> --all:<module>' does not). > >>> Note also that in the above case (stand in D, call 'make --from B') > >>> module > >>> A is not built, regardless of whether there are changes in A or not. > >>> Whereas a simple call to make (still standing in D) would build all > >>> modules that D depends on, directly or indirectly. Thus the options > >>> '--from' and '--since' exist to actively exclude modules from being > >>> built. > >>> > >>> The whole thing becomes a little bit more complicated with multiple > >>> options to '--from' (I never use '--since' and also don't know a valid > >>> use > >>> case so I will ignore it for now) and more complex dependencies then in > >>> the > >>> simple example above. Let's say that if we stand in instsetoo_native > and > >>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This > would > >>> build svx, sfx2 and all modules that depend (directly or indirectly) on > >>> svx > >>> OR sfx2. > >>> > >>> got it, now I just have one problem, why would you not build the > >> dependent > >> modules, if they needed to be built, thats a scenario I dont understand. > >> With a central makefile, <module>/makefile will not be called so we do > not > >> waste cpu cycles. > >> > >> With the .done files, we know when a module was last built and all > modules > >> that depend it should be rebuilt which the rule > >> <module>.done : <module_depend>.done > >> > >> will ensure, so If we have A -> B -> C -> D > >> > >> I go in B, and call make, then when I go in D and make, B,C,D will be > >> made. > >> > >> If we have A -> B -> D C -> D > >> and do the same then only D will be made. > >> > >> So --from is not really saving anything ? > >> > > > > a) In your example you first go into B then, in a second step, into D. > > The '--from' option lets you do the same (well, not really the same, but > > see below) just from D. > > > > b) You go first to B and call make. This makes A, if necessary, then B. > > The making of A is exactly the thing that you want to prevent with the > > '--from' option. Go into D and call 'make --from B'. A is not built. > > > > c) After the discussion with you I am not sure if we still need --from > > because the two reasons I know for its existence my not be relevant with > > the new approach. > > > > c1) With the '--from' option you can tweak the dependency rules at > runtime > > (a bit). This allows you to exclude projects from being built when you > > know that that is not necessary. But from experience I know that can > lead > > to very subtle errors. Letting the system determine what to built is > > usually more reliable. > > > > c2) With build.pl a 'build --all' still builds every module on which the > > one you are standing in depends on. When those modules have been built > > previously, then no compilation takes place. But calling dmake for a > > couple of directories for close to 200 modules (when you stand in > > instsetoo_native) takes a lot of time (several minutes on Windows), even > > when no file has to be compiled. This wasteful way of doing nothing can > be > > prevented with the --from option. > > However, with the new approach and its .done flag files you can determine > > which modules need to be built much faster. You don't have to call dmake > > on directories that where already built. Hm, but this again, does only > > work if your .done files have dependencies on all relevant source files. > > That is something that is missing at least from my script. > > > > > > So, reasons for the existence of '--from' are a result of old/slow > > computers, slow files systems (still valid on Windows), missing global > > dependencies (which we now have for gbuild) and impatient developers. > > > Looking at what Jan has proposed and this discussion so far, I think we > should make some attempt at moving in this direction -- using standard > makefiles. > > Given this discussion, I do understand the additional flexibility of > build.pl, but I also wonder if the flexibility is worth it considering the > IDE tools new developers are likely to be familiar with. > > A complete build will take a while no matter what is used, depending on > your situation. I think this proposal has a lot of merit. >
Thx for the kind words. I have updated the wiki page, to reflect the specific make calls. and R1533869 shows my first version of the central Makefile https://svn.apache.org/repos/asf/openoffice/branches/capstone2013/main/Makefile you will see it contains the intermodule dependencies and nothing more. The setup in every module is kept in a Makefile in the respective module, so we have a clear seperation of module and intermodule dependencies. hope you like it. rgds jan I. > > > > > > > > > > >> > >> While this is easy to do with eg Perl I am not sure how to handle this > >>> with just a Makefile. The straightforward approach with handling > >>> <module>.done files does not work. And that is one of the reasons why > I > >>> don't think that (GNU) makefiles are a good solution for any problem. > >>> Most > >>> of us are used to program object oriented/imperative. Makefiles > require > >>> a > >>> declarative approach. Maybe the use of Perl is not such a bad idea. > >>> Maybe > >>> it would be better to reimplement build.pl with a lot fewer options > and > >>> with better readable code. > >>> > >>> I agree that makefiles are nowhere near a good solution to many of > these > >> problems, but its like windows, I dont like it, but everybody uses it. > >> > >> We could easily write a new build.pl, that also took care of the local > >> makefiles, but our build system would not be in the mainstream, and e.g. > >> the distros would not like to integrate AOO. > >> > >> I have over the last years followed research in building systems, and > >> there > >> are (sadly enough) nobody that tries a real object oriented aproach. > Also > >> if you look at packages like visual studio, QT, eclipse they all use the > >> principle of makefiles with declarative approach. > >> > >> So my simple question is, do we want to approach the main road > (makefiles > >> for unix, visual studio for windows/mac) or do we want to have better > but > >> non standard system. > >> > > > > Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen, > > ach! in meiner Brust" (two souls alas! are dwelling in my breast). The > > pragmatist says to use the make. It is good enough for others, it is > good > > enough for use. > > On the other hand, when I start a new project I usually start with the > > question of what are the best tools for the problem at hand. And make > does > > not seem to be the first or the best answer. Look at our dmake or gbuild > > system. Both don't use make in a standard way. gmake even defines its > own > > language, object oriented and imperative, on top of the GNU make macros. > > That is, for me, an act of desperation. > > I have made experiments with an alternative approach, a domain specific > > language somewhat similar to Java. I personally like this approach > because > > a) it uses the paradigm that I already use when writing C++ code. That > > means that I can apply my existing knowledge to the build process and > that > > I don't have to remember all the tricks and pitfalls of makefiles. > > b) as expected it was much easier to handle file dependencies and > parallel > > processing of build jobs in Java than adding object orientation and > > imperative control flow to makefiles. > > > > If I had the time and if I would be the one working on it then I would > > prefer an non-Makefile approach. But maybe I am just suffering too much > > from one of the 'three great virtues of a programmer': hubris. > > > > You are the one who leads the build project changes, so you have to > decide > > which approach to use. I am not trying to make your life harder (than > > necessary), I am only trying to point out some of the pitfalls that I > have > > encountered in the past. And to prevent you from removing features that > I > > use :-) > > > > > > > > > >> rgds > >> jan I. > >> > >> Ps. its always refreshing to discuss with you, you often have an > >> alternative approach, which is not just a dream but doable. > >> > > > > Thanks. That makes two of us. > > > > Have a nice weekend, > > Andre > > > > > > > >> > >> -Andre > >>> > >>> > >>> --prepare > >>>> > >>>>> Also one option that is important for our every day work. Use > >>>>> case: > >>>>> You make changes in <module> and are not sure if these changes are > >>>>> compatible/incompatible. To be on the safe side you discard the > output > >>>>> of > >>>>> all depending modules. To save time you keep the output of all other > >>>>> modules. > >>>>> > >>>>> Often used together with '--from' like 'make --prepare --from > >>>>> svx' to > >>>>> prepare a build after making changes in svx. > >>>>> > >>>>> Documentation changed, funny thing is that svx does not clear > >>>>> correctly > >>>>> > >>>> on > >>>> my ubuntu build. > >>>> > >>>> > >>>> --since <module> > >>>> > >>>>> A variant of '--from'. The only difference is that <module> > >>>>> itself > >>>>> is > >>>>> not built. > >>>>> > >>>>> If your proposed approach is similar to what my script produces > >>>>> then > >>>>> it > >>>>> is not too difficult to support --from/--since. I made some > >>>>> experiments > >>>>> in > >>>>> this direction but was to lazy to finish them. > >>>>> > >>>>> My approach is very similar, but I failed to see how --since is > >>>>> > >>>> supported. > >>>> And question is if its real important. > >>>> > >>>> > >>>> --job > >>>> > >>>>> --pre_job > >>>>> --post_job > >>>>> These are sometimes handy to run a non-standard command for all > >>>>> modules. > >>>>> > >>>>> I have added them, they are by the way a good example why we need a > >>>>> > >>>> discussion I have never used them. > >>>> > >>>> However maybe the real discussion is "do we want to replace build and > >>>> have > >>>> a main/Makefile instead?" > >>>> > >>>> > >>>> > >>>> - I have not used the rest of the unsupported options and would not > >>>> miss > >>>> > >>>>> them. Others may have other sets of options that are important to > >>>>> them. > >>>>> > >>>>> > >>>>> Some general remarks: > >>>>> > >>>>> - Why keep one makefile per module? Why not put all the inter-module > >>>>> dependencies into one file (like my script does)? > >>>>> > >>>>> Ups, I did not explain that correctly, I propose 1 Makefile > >>>>> > >>>> "main/Makefile" > >>>> with all inter-module and 1 Makefile "<module>/Makefile" that today > just > >>>> will call the old makefiles as described in prj/build.lst > >>>> > >>>> - Why not use the oportunity to move (a part of) the build environment > >>>> out > >>>> > >>>> of the way to, say, build/ ? > >>>>> > >>>>> You have guessed my next step. > >>>>> > >>>> > >>>> - How are dependencies between modules handled (just the manual > >>>> > >>>>> dependencies from prj/build.lst or also the file dependencies > >>>>> introduced > >>>>> by > >>>>> gmake). > >>>>> > >>>>> See doc. on --from. Its done with <module>.done files > >>>>> > >>>> > >>>> - How is the output of the individual calls to dmake or GNU make > >>>> > >>>>> handled/made accessible. Ie. if there is a build error, how can I > look > >>>>> up > >>>>> the corresponding build output? > >>>>> > >>>>> see doc. script make_log > >>>>> > >>>> - Are the gmake makefiles included (run in the same process) or is > GNU > >>>> > >>>>> make started for them it its own process? > >>>>> > >>>>> For a start they would be called (own process), but its something > >>>>> where > >>>>> > >>>> I > >>>> have no strong opinions. > >>>> > >>>> Please (just to be sure), this proposal has nothing to do with the > >>>> students > >>>> work, its simply because I saw a positive discussion on removing > >>>> build.pl > >>>> , > >>>> and spent a couple of hours looking at it. If there is a preference > not > >>>> to > >>>> remove build.pl I will simply forget it. > >>>> > >>>> rgds > >>>> jan I. > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> Regards, > >>>>> Andre > >>>>> > >>>>> > >>>>> > >>>>> It has been roughly tested it, thanks to a clever utility from > andre. > >>>>> > >>>>>> As discussed build.pl contains a lot of options, which need to be > >>>>>> considered in a makefile. > >>>>>> > >>>>>> My suggestion is on > >>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**< > http://wiki.openoffice.org/****wiki/Build_System_Analysis:**> > >>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**< > http://wiki.openoffice.org/**wiki/Build_System_Analysis:**> > >>>>>> > > >>>>>> build.pl_versus_makefile<http:****//wiki.openoffice.org/wiki/****< > http://wiki.openoffice.org/wiki/**> > >>>>>> Build_System_Analysis:build.****pl_versus_makefile<http://** > >>>>>> wiki.openoffice.org/wiki/**Build_System_Analysis:build.** > >>>>>> pl_versus_makefile< > http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile > > > >>>>>> > > >>>>>> > >>>>>> Please feel free to edit/comment on the page. I have reduced to > >>>>>> options > >>>>>> a > >>>>>> lot, and some of them might be in use. > >>>>>> > >>>>>> thanks in advance for your comments. > >>>>>> > >>>>>> > >>>>>> ------------------------------******--------------------------** > >>>>>> --** > >>>>>> > >>>>> --**--------- > >>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org< > >>>>> http://apache.org**> > >>>>> <dev-unsubscribe@**openoffice.**apache.org< > http://openoffice.apache.org> > >>>>> <dev-unsubscribe@**openoffice.apache.org< > dev-unsubscr...@openoffice.apache.org> > >>>>> > > >>>>> For additional commands, e-mail: dev-h...@openoffice.apache.org > >>>>> > >>>>> > >>>>> > >>>>> ------------------------------****----------------------------** > >>> --**--------- > >>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.**a**pache.org< > http://apache.org> > >>> <dev-unsubscribe@**openoffice.apache.org< > dev-unsubscr...@openoffice.apache.org> > >>> > > >>> For additional commands, e-mail: dev-h...@openoffice.apache.org > >>> > >>> > >>> > > > > ------------------------------**------------------------------**--------- > > To unsubscribe, e-mail: dev-unsubscribe@openoffice.**apache.org< > dev-unsubscr...@openoffice.apache.org> > > > > For additional commands, e-mail: dev-h...@openoffice.apache.org > > > > > > > -- > > ------------------------------------------------------------------------------------------------- > MzK > > "Truth is stranger than fiction, but it is because Fiction is obliged > to stick to possibilities. Truth isn't." > -- "Following the Equator", Mark Twain >
