Keeping it simple, is a great idea. I am not sure if introducing parallelism is a good idea or if it's easy to implement and maintain.
If it is design is modular, I think all these can be added as plugins. Performance ??!! On Sat, Feb 18, 2012 at 1:47 PM, Bruce Atherton <br...@callenish.com> wrote: > This too I find a great idea. Multicores mean we need more ways of > exploiting parallelism, particularly if they can be identified automatically > by the application. > > For backward compatibility it would have to be optional, though, either > specified on the command line or at the build file level or using a > different kind of target tag. Of course, the additional information you > suggest on targets would make that the case anyway, but in some cases I > think we might be able to automate it based just on what we have if the > build system is written properly so that dependencies on targets that > provide needed resources are explicitly identified. > > Too many build systems in my experience rely on the order the dependencies > are resolved on a higher order target rather than explicitly identifying > dependencies on the targets that they are required on. This despite the fact > that we have some language somewhere that you can't rely on the order of > resolution of the dependencies. Sometimes that has proved a requirement to > avoid targets that depended on each other, but I've seen it used as a > shortcut instead far too often. > > Creating Ant clusters is also a great idea, at least to plan for. Perhaps > something like Zookeeper to coordinate builds and a message bus like > ActiveMQ or perhaps better QPid to schedule and report back to other nodes > on the results. > > The idea of turning the dependency tree into a state machine is also > interesting. It would combine the tree and the resolving of the tree with > execution of tasks into a single entity. I worry, though, that some > flexibility in our current system might be lost if the two portions are > combined into one. Perhaps not. It could also introduce the possibility of > cycles. Currently the dependency tree gets executed as a DAG but there have > been a lot of requests for looping which a state machine could more easily > support. > > On 2/18/2012 2:02 AM, Gilles Scokart wrote: >> >> For me, one feature for a 2,0 would be a different style of dependency >> tree that would allow better parallel execution (on the same machine, >> or why not on distributed machines). >> I see the 'targets' being more declarative, becoming a state >> transition saying : I need this resources in that state, I will use >> this other resources (and I don't want the to change during my >> execution, and I will produce this other resources in that other >> state. >> >> The dependency tree would be an logical engine finding the shortest >> path to go to the desired state, using parallel/distributed processing >> when possible. >> >> That's what I miss with existing build system : I want to go as >> quickly as possible to a desired build state (from a current state). >> >> >> >> Gilles Scokart >> >> >> >> On 17 February 2012 20:07, Bruce Atherton<br...@callenish.com> wrote: >>> >>> It doesn't require a rewrite, but a rewrite could simplify integrating a >>> usecase like this as well as integrating other features that we already >>> have >>> into it and making them simpler and unified inthe code. I agree the >>> usecase >>> is an excellent one which could simplify the lives of exactly the type of >>> users I am talking about. >>> >>> It sounds like you are suggesting that the dependency tree be extensible >>> and >>> modifiable, perhaps manipulable, within targets as well so long as that >>> part >>> of the tree hasn't run yet. In a sense that is what macros do because >>> they >>> allow you to swap in some static block of tasks to replace a single task. >>> There is also the feature from EasyAnt for changing target dependencies. >>> But >>> what I'm hearing is that you want more flexibility than that. >>> >>> Something to walk the existing dependency tree, perhaps, with conditional >>> behaviors to modify the metadata on existing element such as dependencies >>> and if/unless, replacing the element with another or a subtree (perhaps >>> itself dynamically walked and created), adding branches, perhaps deleting >>> elements or subtrees. Kind of like what we can do with a tree of files >>> and >>> directories already. Does that sound like what the design you'd like to >>> see >>> would have? >>> >>> And perhaps it could encompass providing both the macro and target >>> dependency changes to the tree as well, along with any other code we have >>> that alters the dependency tree. I'm not sure which of the various ways >>> to >>> call back into Ant do this. I'm sure there are other examples in the >>> codebase. >>> >>> I'm not familiar enough with this part of the code any more to know >>> whether >>> there is already a single elegant solution in Ant 1 that all the code >>> which >>> modifies the dependency tree shares, but given our BC requirements I >>> doubt >>> it. >>> >>> One example of a FileSystemProvider that Java 7 suggests in its API docs >>> is >>> a "memory" file system, one identified by the URL "memory://". Perhaps >>> our >>> dependency tree could be a kind of file system, then we could reuse vast >>> swathes of code we already have, both in the standard class libraries and >>> in >>> Ant itself. Just an idea off the top of my head. >>> >>> >>> On 2/17/2012 5:53 AM, Dominique Devienne wrote: >>>> >>>> 2012/2/17 Bruce Atherton<br...@callenish.com>: >>>>> >>>>> A lot of companies have their own, internally written build file >>>>> generators >>>>> just so their build systems are consistent and exactly what they want. >>>>> Our >>>>> Related Projects and External Tools page has some of these that were >>>>> made >>>>> public, I suspect. >>>>> >>>>> Surely there is a better way than requiring users of Ant to write >>>>> generators >>>>> to deal with the complexity and keep it customized. >>>> >>>> At one point I did write a build(s) (XSLT-based) generator >>>> specifically for a large and hairy project. Later I rewrote the whole >>>> thing with macrodefs. But my point is that I don't view build >>>> generators as bad, in fact it often helps IMHO to have a declarative >>>> custom DSL (in XML in my case, so read "DSL" with a grain of salt) >>>> that's used as the input for generating Ant build fragments, and have >>>> those fragments be able to "insert" them into the target graph. I've >>>> also long felt Ant needed generalized if/unless/os (and my own >>>> extensions like ifTrue, unlessTrue, when) on any "XML tag" (or >>>> UnknownElement if you prefer), just read the recent "add if/unless to >>>> <javac>'s<compilerarg>" thread.<macrodef> is nice, but you can't use >>>> it for arbitrary, *and conditional*, XML "fragments" inside tasks. All >>>> those things you can often do more easily with a generator, but that's >>>> often cumbersome, doesn't play well with IDEs, etc... I guess I'm >>>> saying I've often wished for generator-like features as a built-in >>>> part of Ant. Do you see what I'm saying? Ant now does late >>>> "conversion" from UnkownElement to actual configuration of the Java >>>> code it maps to, and a way to influence/transform that almost AST-like >>>> graph would make Ant more powerful and flexible, perhaps at the >>>> expense of creating "dialects" unreadable to someone not familiar with >>>> them. Given Ant's XML roots, perhaps a tighter built-in integration >>>> with XSLT to dynamically "rewrite" the build at runtime/buildtime >>>> would be one way to achieve what I envision (notwithstanding the talks >>>> of non-XML front-ends of course). >>>> >>>> Stepping of my soapbox now :) What I'm saying has nothing to do with >>>> Java7, nor necessarily require a rewrite either. --DD >>>> >>>> --------------------------------------------------------------------- >>>> To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org >>>> For additional commands, e-mail: dev-h...@ant.apache.org >>>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org >>> For additional commands, e-mail: dev-h...@ant.apache.org >>> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org >> For additional commands, e-mail: dev-h...@ant.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org > For additional commands, e-mail: dev-h...@ant.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@ant.apache.org For additional commands, e-mail: dev-h...@ant.apache.org
