On Fri, Oct 18, 2013 at 7:52 AM, Andre Fischer <awf....@gmail.com> wrote:
> On 18.10.2013 15:58, janI wrote: > >> On 18 October 2013 15:00, Andre Fischer <awf....@gmail.com> wrote: >> >> On 18.10.2013 14:02, janI wrote: >>> >>> sd >>>> >>>> >>>> On 18 October 2013 13:36, Andre Fischer <awf....@gmail.com> wrote: >>>> >>>> On 18.10.2013 11:32, janI wrote: >>>> >>>>> Hi. >>>>> >>>>>> due to the discussion in thread "Mentor a new build system", I have >>>>>> made a >>>>>> proposal for a central Makefile located in main. >>>>>> >>>>>> Hi Jan, >>>>>> >>>>> it is great that you are going to improve this part of the build >>>>> system. >>>>> But I think that we need more details about how the proposed build >>>>> system >>>>> works. Without them I can not really evaluate the proposal. >>>>> >>>>> First of all, I agree with juergens remarks that this should be >>>>> >>>> discussed >>>> before implemented, hence the wiki page. >>>> >>>> Secondly this has nothing directly to do with the proposed build system, >>>> its a simple replacement of build.pl in the current system. >>>> >>>> Yes, that is how I understood it. I just did not know how to call the >>> build.pl replacement. >>> >>> >>> >>> I know that build.pl works, but having a Makefile in main, would make >>>> us >>>> one step closer on being compatible with the distros. To me this job is >>>> a >>>> simple cleanup, not something we deadly need, but nice to have. >>>> >>>> >>>> Some remarks regarding the missing options: >>>> >>>>> --from <module> >>>>> This is one of the more important options and one that I use >>>>> frequently >>>>> (also in the form --all:<module>). >>>>> Note that if you are in <moduleA> and call 'make --from <moduleB>' >>>>> then >>>>> all modules are built >>>>> a) which <moduleA> depends on >>>>> b) but not those that <moduleB> depends on >>>>> c) Both <moduleA> and <moduleB> are built. >>>>> >>>>> I have changed the documentation. >>>>> >>>> I use the --all:<module> myself very often, and have changed the >>>> documentation, because it is of course supported. >>>> >>>> The difference is that you do the call in main, but that is a minor >>>> detail >>>> that can be easily corrected (have <module>/Makefile calling >>>> main/Makefile. >>>> >>>> I have also changed documentation on --html due to juergens comments. >>>> >>>> I am not sure that we understand --from and --since in the same way so >>> I >>> will try to explain what I think they do. >>> >>> Let's imagine that we have a simple project with modules A, B, C, D and >>> E. >>> where B depends on A, C on B, D on C, and E on D. >>> A ' make all' would mean 'make E'. The dependencies would then lead to >>> building modules A, B, C, D, E in this order. >>> If I am in E and call 'make --from C' then only C, D, and E should be >>> built. A 'make --since C' would only build D and E. >>> >>> If I am in D and call 'make --from B' then modules B, C, and D are built. >>> Call 'make --since B' to build only C and D. >>> Note that 'make --from' accepts more than one module name (while 'make >>> --all:<module>' does not). >>> Note also that in the above case (stand in D, call 'make --from B') >>> module >>> A is not built, regardless of whether there are changes in A or not. >>> Whereas a simple call to make (still standing in D) would build all >>> modules that D depends on, directly or indirectly. Thus the options >>> '--from' and '--since' exist to actively exclude modules from being >>> built. >>> >>> The whole thing becomes a little bit more complicated with multiple >>> options to '--from' (I never use '--since' and also don't know a valid >>> use >>> case so I will ignore it for now) and more complex dependencies then in >>> the >>> simple example above. Let's say that if we stand in instsetoo_native and >>> call 'make --from svx sfx2'. Note that svx depends on sfx2. This would >>> build svx, sfx2 and all modules that depend (directly or indirectly) on >>> svx >>> OR sfx2. >>> >>> got it, now I just have one problem, why would you not build the >> dependent >> modules, if they needed to be built, thats a scenario I dont understand. >> With a central makefile, <module>/makefile will not be called so we do not >> waste cpu cycles. >> >> With the .done files, we know when a module was last built and all modules >> that depend it should be rebuilt which the rule >> <module>.done : <module_depend>.done >> >> will ensure, so If we have A -> B -> C -> D >> >> I go in B, and call make, then when I go in D and make, B,C,D will be >> made. >> >> If we have A -> B -> D C -> D >> and do the same then only D will be made. >> >> So --from is not really saving anything ? >> > > a) In your example you first go into B then, in a second step, into D. > The '--from' option lets you do the same (well, not really the same, but > see below) just from D. > > b) You go first to B and call make. This makes A, if necessary, then B. > The making of A is exactly the thing that you want to prevent with the > '--from' option. Go into D and call 'make --from B'. A is not built. > > c) After the discussion with you I am not sure if we still need --from > because the two reasons I know for its existence my not be relevant with > the new approach. > > c1) With the '--from' option you can tweak the dependency rules at runtime > (a bit). This allows you to exclude projects from being built when you > know that that is not necessary. But from experience I know that can lead > to very subtle errors. Letting the system determine what to built is > usually more reliable. > > c2) With build.pl a 'build --all' still builds every module on which the > one you are standing in depends on. When those modules have been built > previously, then no compilation takes place. But calling dmake for a > couple of directories for close to 200 modules (when you stand in > instsetoo_native) takes a lot of time (several minutes on Windows), even > when no file has to be compiled. This wasteful way of doing nothing can be > prevented with the --from option. > However, with the new approach and its .done flag files you can determine > which modules need to be built much faster. You don't have to call dmake > on directories that where already built. Hm, but this again, does only > work if your .done files have dependencies on all relevant source files. > That is something that is missing at least from my script. > > > So, reasons for the existence of '--from' are a result of old/slow > computers, slow files systems (still valid on Windows), missing global > dependencies (which we now have for gbuild) and impatient developers. Looking at what Jan has proposed and this discussion so far, I think we should make some attempt at moving in this direction -- using standard makefiles. Given this discussion, I do understand the additional flexibility of build.pl, but I also wonder if the flexibility is worth it considering the IDE tools new developers are likely to be familiar with. A complete build will take a while no matter what is used, depending on your situation. I think this proposal has a lot of merit. > > >> >> While this is easy to do with eg Perl I am not sure how to handle this >>> with just a Makefile. The straightforward approach with handling >>> <module>.done files does not work. And that is one of the reasons why I >>> don't think that (GNU) makefiles are a good solution for any problem. >>> Most >>> of us are used to program object oriented/imperative. Makefiles require >>> a >>> declarative approach. Maybe the use of Perl is not such a bad idea. >>> Maybe >>> it would be better to reimplement build.pl with a lot fewer options and >>> with better readable code. >>> >>> I agree that makefiles are nowhere near a good solution to many of these >> problems, but its like windows, I dont like it, but everybody uses it. >> >> We could easily write a new build.pl, that also took care of the local >> makefiles, but our build system would not be in the mainstream, and e.g. >> the distros would not like to integrate AOO. >> >> I have over the last years followed research in building systems, and >> there >> are (sadly enough) nobody that tries a real object oriented aproach. Also >> if you look at packages like visual studio, QT, eclipse they all use the >> principle of makefiles with declarative approach. >> >> So my simple question is, do we want to approach the main road (makefiles >> for unix, visual studio for windows/mac) or do we want to have better but >> non standard system. >> > > Good analysis. Maybe I should answer with Faust: "Zwei Seelen wohnen, > ach! in meiner Brust" (two souls alas! are dwelling in my breast). The > pragmatist says to use the make. It is good enough for others, it is good > enough for use. > On the other hand, when I start a new project I usually start with the > question of what are the best tools for the problem at hand. And make does > not seem to be the first or the best answer. Look at our dmake or gbuild > system. Both don't use make in a standard way. gmake even defines its own > language, object oriented and imperative, on top of the GNU make macros. > That is, for me, an act of desperation. > I have made experiments with an alternative approach, a domain specific > language somewhat similar to Java. I personally like this approach because > a) it uses the paradigm that I already use when writing C++ code. That > means that I can apply my existing knowledge to the build process and that > I don't have to remember all the tricks and pitfalls of makefiles. > b) as expected it was much easier to handle file dependencies and parallel > processing of build jobs in Java than adding object orientation and > imperative control flow to makefiles. > > If I had the time and if I would be the one working on it then I would > prefer an non-Makefile approach. But maybe I am just suffering too much > from one of the 'three great virtues of a programmer': hubris. > > You are the one who leads the build project changes, so you have to decide > which approach to use. I am not trying to make your life harder (than > necessary), I am only trying to point out some of the pitfalls that I have > encountered in the past. And to prevent you from removing features that I > use :-) > > > > >> rgds >> jan I. >> >> Ps. its always refreshing to discuss with you, you often have an >> alternative approach, which is not just a dream but doable. >> > > Thanks. That makes two of us. > > Have a nice weekend, > Andre > > > >> >> -Andre >>> >>> >>> --prepare >>>> >>>>> Also one option that is important for our every day work. Use >>>>> case: >>>>> You make changes in <module> and are not sure if these changes are >>>>> compatible/incompatible. To be on the safe side you discard the output >>>>> of >>>>> all depending modules. To save time you keep the output of all other >>>>> modules. >>>>> >>>>> Often used together with '--from' like 'make --prepare --from >>>>> svx' to >>>>> prepare a build after making changes in svx. >>>>> >>>>> Documentation changed, funny thing is that svx does not clear >>>>> correctly >>>>> >>>> on >>>> my ubuntu build. >>>> >>>> >>>> --since <module> >>>> >>>>> A variant of '--from'. The only difference is that <module> >>>>> itself >>>>> is >>>>> not built. >>>>> >>>>> If your proposed approach is similar to what my script produces >>>>> then >>>>> it >>>>> is not too difficult to support --from/--since. I made some >>>>> experiments >>>>> in >>>>> this direction but was to lazy to finish them. >>>>> >>>>> My approach is very similar, but I failed to see how --since is >>>>> >>>> supported. >>>> And question is if its real important. >>>> >>>> >>>> --job >>>> >>>>> --pre_job >>>>> --post_job >>>>> These are sometimes handy to run a non-standard command for all >>>>> modules. >>>>> >>>>> I have added them, they are by the way a good example why we need a >>>>> >>>> discussion I have never used them. >>>> >>>> However maybe the real discussion is "do we want to replace build and >>>> have >>>> a main/Makefile instead?" >>>> >>>> >>>> >>>> - I have not used the rest of the unsupported options and would not >>>> miss >>>> >>>>> them. Others may have other sets of options that are important to >>>>> them. >>>>> >>>>> >>>>> Some general remarks: >>>>> >>>>> - Why keep one makefile per module? Why not put all the inter-module >>>>> dependencies into one file (like my script does)? >>>>> >>>>> Ups, I did not explain that correctly, I propose 1 Makefile >>>>> >>>> "main/Makefile" >>>> with all inter-module and 1 Makefile "<module>/Makefile" that today just >>>> will call the old makefiles as described in prj/build.lst >>>> >>>> - Why not use the oportunity to move (a part of) the build environment >>>> out >>>> >>>> of the way to, say, build/ ? >>>>> >>>>> You have guessed my next step. >>>>> >>>> >>>> - How are dependencies between modules handled (just the manual >>>> >>>>> dependencies from prj/build.lst or also the file dependencies >>>>> introduced >>>>> by >>>>> gmake). >>>>> >>>>> See doc. on --from. Its done with <module>.done files >>>>> >>>> >>>> - How is the output of the individual calls to dmake or GNU make >>>> >>>>> handled/made accessible. Ie. if there is a build error, how can I look >>>>> up >>>>> the corresponding build output? >>>>> >>>>> see doc. script make_log >>>>> >>>> - Are the gmake makefiles included (run in the same process) or is GNU >>>> >>>>> make started for them it its own process? >>>>> >>>>> For a start they would be called (own process), but its something >>>>> where >>>>> >>>> I >>>> have no strong opinions. >>>> >>>> Please (just to be sure), this proposal has nothing to do with the >>>> students >>>> work, its simply because I saw a positive discussion on removing >>>> build.pl >>>> , >>>> and spent a couple of hours looking at it. If there is a preference not >>>> to >>>> remove build.pl I will simply forget it. >>>> >>>> rgds >>>> jan I. >>>> >>>> >>>> >>>> >>>> >>>> Regards, >>>>> Andre >>>>> >>>>> >>>>> >>>>> It has been roughly tested it, thanks to a clever utility from andre. >>>>> >>>>>> As discussed build.pl contains a lot of options, which need to be >>>>>> considered in a makefile. >>>>>> >>>>>> My suggestion is on >>>>>> http://wiki.openoffice.org/******wiki/Build_System_Analysis:**<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**> >>>>>> **<http://wiki.openoffice.org/****wiki/Build_System_Analysis:**<http://wiki.openoffice.org/**wiki/Build_System_Analysis:**> >>>>>> > >>>>>> build.pl_versus_makefile<http:****//wiki.openoffice.org/wiki/****<http://wiki.openoffice.org/wiki/**> >>>>>> Build_System_Analysis:build.****pl_versus_makefile<http://** >>>>>> wiki.openoffice.org/wiki/**Build_System_Analysis:build.** >>>>>> pl_versus_makefile<http://wiki.openoffice.org/wiki/Build_System_Analysis:build.pl_versus_makefile> >>>>>> > >>>>>> >>>>>> Please feel free to edit/comment on the page. I have reduced to >>>>>> options >>>>>> a >>>>>> lot, and some of them might be in use. >>>>>> >>>>>> thanks in advance for your comments. >>>>>> >>>>>> >>>>>> ------------------------------******--------------------------** >>>>>> --** >>>>>> >>>>> --**--------- >>>>> To unsubscribe, e-mail: dev-unsubscribe@openoffice.****a**pache.org< >>>>> http://apache.org**> >>>>> <dev-unsubscribe@**openoffice.**apache.org<http://openoffice.apache.org> >>>>> <dev-unsubscribe@**openoffice.apache.org<dev-unsubscr...@openoffice.apache.org> >>>>> > >>>>> For additional commands, e-mail: dev-h...@openoffice.apache.org >>>>> >>>>> >>>>> >>>>> ------------------------------****----------------------------** >>> --**--------- >>> To unsubscribe, e-mail: >>> dev-unsubscribe@openoffice.**a**pache.org<http://apache.org> >>> <dev-unsubscribe@**openoffice.apache.org<dev-unsubscr...@openoffice.apache.org> >>> > >>> For additional commands, e-mail: dev-h...@openoffice.apache.org >>> >>> >>> > > ------------------------------**------------------------------**--------- > To unsubscribe, e-mail: > dev-unsubscribe@openoffice.**apache.org<dev-unsubscr...@openoffice.apache.org> > > For additional commands, e-mail: dev-h...@openoffice.apache.org > > -- ------------------------------------------------------------------------------------------------- MzK "Truth is stranger than fiction, but it is because Fiction is obliged to stick to possibilities. Truth isn't." -- "Following the Equator", Mark Twain
