On 1/4/13 2:56 PM, Alex J Lennon wrote:
Can anybody advise on whether bitbake currently supports offloading of build tasks onto multiple systems? Perhaps cloud based?
I see Chris Larson has already answer the main question here.. but I've got some additional insight I can add.
I'm thinking that it would be more efficient for me if I could bring up a number of Amazon EC2 servers (or similar) then have bitbake parallelise the build onto those servers to significantly reduce my build times?
In the past I've had people ask me questions about remote builds, parallel building packages, etc. In the past when I've attempted to figure out if it would be useful, the cost of the data transfer time always comes into play. How long does it take to simply perform the task locally vs go to the remote machine, perform the task, and get the data back. For a lot of the tasks that OE does, the remote tasks will end up being slower on a reasonably modern i7 (or similar Xeon) machine. I'm not sure using something like a cloud server would end up helping due to this.
From experience I'd say that if anything you'd likely want to modify the bitbake scheduler with some type of locality information. I.e. if you extract the sources on a node, you most likely want to patch, configure, compile, install, QA and package on that node as well.... but that also means a way to establish node specific tmp/work space is needed for performance as well.
I see bitbake supports a level of task parallelisation on a single box. Can parallelisation of build onto multiple systems be achieved?
All of the above are possible, but I suspect there is a lot of design and refactoring required to come up with a system that is both efficient and fast.
Is it something that should even be a goal?
It's not currently a goal. But it does get asked about every now and then. It would be nice to have something more then anecdotal information about if it would be a good idea or not. This is the type of thing that I think would make an excellent research project. (Hey and if it makes things able to be built for more people, faster, and cheaper great! If not, we'd have a more informed response to questions like this.)
--Mark
Cheers, Alex _______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
_______________________________________________ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
