It's a multifaceted issue it's becoming a bottleneck to support the growing list of committers.
The build has issues like python2, python 3, c++ being built serially in containers separately before being copied to a destination container. The integration tests need an hour to build an image even before they start running. Moving the docs to a separate repo gives us some short term wins and is a good start. It will unblock committers just concerned with doc updates and we can enable a near instantaneous update to the website for preview. On Thu, Sep 26, 2019 at 12:51 AM Ivan Kelly <iv...@apache.org> wrote: > > If we believe coordinating multiple repos is too difficult. We can > > consider migrate to bazel as a build system with a hosted cache. That can > > give us build speed benefits while maintaining the mono repo structure. > > What is the problem motivating this change? From this comment it seems > that it's the time it takes to build the whole tree, but moving > docs/c++ wouldn't have much effect on this. > > While I would 100% support using bazel, especially since there's > multiple languages in the project now. However, I think that's a > harder change to get buyin on and i don't think that solves the issue > here. > > I large amount of the time of the build seems to be lost to shading > and copying jars around. There's also some things that take a stupidly > long time. > <snip> > ... > [INFO] Pulsar Kafka compatibility :: API .................. SUCCESS [ > 18.720 s] > [INFO] Pulsar Kafka compatibility 0.8 :: API .............. SUCCESS [ > 20.485 s] > [INFO] Pulsar Kafka compatibility 0.9 :: API .............. SUCCESS [ > 16.250 s] > ... > [INFO] Pulsar IO :: RabbitMQ .............................. SUCCESS [ > 28.521 s] > ... > [INFO] Pulsar Examples :: Flink ........................... SUCCESS [ > 12.836 s] > ... > </snip> > > We'd get an immediate win on speed if we stopped shading so much and > making unnecessary copies. > > -Ivan >
