On 19/08/15 12:34 +0100, Robbie Gemmell wrote:
I can see certain benefits to such a separation, mainly for folks interested only in the bindings, but if I'm honest I'm not sure those outweigh the additional complication it seems it may bring in some of the other areas.
I think they actually do but probably because I don't see any complications being added to proton-c itself but rather a simplification of the current structure. It'll bring a more intuitive structure to the project rather than having the bindings under `proton-c`, which is very confusing :)
The python bindings are slightly more interesting than the others due to being at the heart of the python based tests that exist for proton-c and proton-j, so splitting them out actually creates a bit of a circular dependency with proton / its tests (especially when having the submodules in both repos). You mentioned thinking we wouldn't need to track the binding within the proton tree (but could do so to prevent a breaking change for existing things), and instead use an installed version of the bindings. In practice I guess it would be about the same in terms of updating proton-c and the bindings and the related python-based tests at the same time (adding an install, or possibly two, in the middle).
It won't be the same. The reason is that we're currently ensuring that proton-c doesn't break the bindings 'master' version. This is being done on the assumption that bindings will be released along with proton-c and there'll be a binding release per every proton-c minor. The above is not an ideal scenarion in my opinion. Ideally, we'd have a release of the bindings that will be able to track several minor releases of proton-c, but this will probably come later once proton-c stabilizes a bit more. However, I believe the current structure allows for breaking changes to land because it tests things against master only rather than making sure older, still maintained, versions are not being broken. Going back to the proposal, I think the circular dependency is not really necessary and we could just have the python bindings being installed. The installed version can either be the current master branch (pip knows how to install from git) or the latest stable version. Furthermore, we could even have a list of versions that we want to test against. To expand the above issue a bit more, I think it's wrong to have the python tests as part of the in-tree tests for proton-c. Ideally, this should be the work of a CI system responsible for doing integration tests. Not saying we should solve this issue now but the proposed structure allows for it to happen.
Releasing (in terms of source, here at Apache) seems like it would become more 'interesting' though, with potential again for the bindings needing proton-c (+proton-j) updates, but proton-c (+proton-j) needing the bindings updates for updated tests. I guess that might point to splitting the python tests out on their own as well so it depended on both separately.
What python tests are you referring to? The ones under `tests/python` ? Those are the only ones I know of and those are still in the `proton-c` repo. The python bindings access those through the submodule for now but yes, I'd agree that eventually, those tests could even be on their own.
Perhaps I'm overthinking this though; how would you see some of those things working?
Totally honest? I think Java should have its own tests and let python alone. Maintaining python2.5 - because that's what the tests support - plus all the self-written tests libraries and whatnot is a burden that I think we should not carry. Most of the things being maintained under the `tests/python` 'package' exists already in the python community and are widely used and tests. I know it seems easier to maintain 1 test suite for 2 bindings rather than 2 but it really isn't. Not in this case, at the very least. However, based on the current situation, I'd either have the test code on its own or simply follow the proposed structure and have the bindings following proton-c and consuming tests from there.
I also can't say that I'd be a particular fan of using submodules, most of the things I've read about or tried with them have been a turnoff...but I'm mostly skipping over that for now.
In my experience, if done right, they work and they're not hard to update/maintain. Circular dependencies are to avoid, agreed. One case where I've had good experience with submodules is the rustlang. I guess giving this a try would tell better. Thanks for your reply, Flavio
Robbie On 18 August 2015 at 08:17, Flavio Percoco <fla...@redhat.com> wrote:Greetings, I'd like to take a chance, now that 0.10 is out, to make a point in favor of using submodules rather than having all bindings in the same repository. I think qpid-proton has reached the point where it's mature enough to have some of the bindings promoted to having their own repos. More importantly, I believe these bindings are mature enough and ready to build a community on their own. Lets take the python bindings, for instance. I've been contributing to the python bindings quite a bit in the last couple of months. The bindings went from depending on cmake for tests to having their own, still dependant on proton-test, test suite that is self-maintained. Furthermore, the python bindings went from depending on cmake to be built to being self-built using python's internal build system. It's still possible - and in fact, it still happens - to build these drivers running cmake, although it's, in my personal opinion, not recommended. As you can see from the above, the python bindings depend on proton-c but they don't need to live in the same codebase. Therefore, I've prepared a small POC to show case this repo structure: - https://github.com/FlaPer87/test-python-qpid-proton The above repo contains the python-qpid-proton code. As you can tell from the commit history, not many changes were required to set it up. The relevant bits of this repo, I believe, are the ones related to the submodule. The qpid-proton submodule points qpid-proton's master branch in the test-python-qpid-proton branch, whereas in the 0.10 branch, it tracks qpid-proton's 0.10.x. This is important for the bindings to keep track of the right code/release/tests. - https://github.com/FlaPer87/qpid-proton/tree/submodules The above is a fork of the qpid-proton repo we use already but rather than having the python-qpid-bindings in tree, it just tracks the master branch. (I created a `submodules` branch because I didn't want to mess with the master branch in my fork). I don't think tracking the binding in the qpid-proton repo is necessary but I see how it's useful to avoid introducing breaking changes on drivers. To be more precise, I don't think the above is necessary because if we are testing bindings in cmake to make sure we don't break backwards compatibility, we could simply install the latest stable version and test against that, rather than testing against the latest binding's master, which could, but shouldn't, be broken. I wouldn't probably advocate for the above to applied to every binding we have since some of them might not be ready for this but I definitely want to advocate to make this happen for the python binding (and other bindings). This structure will make it easier to mainting bindings, to build a community around these bindings - I contribute more to the python binding than proton-c. It'll also help keeping a clean history for each project and to issue binding releases, therefore creating tags for bindings, as needed. I get it's worked well so far and probably some of you won't be happy with this proposal but, git is powerful enough to support the above structure, which is ideal for these kind of projects. Correct me if I'm wrong but I believe the current structure also comes from the old times when qpid-proton used to be in subversion. Hope the above makes sense and I'm sorry for such a long email, I should've drunk less coffee. Flavio P.S: I'm more than happy to help setting this up and to bring it back if the experiment fails. -- @flaper87 Flavio Percoco
-- @flaper87 Flavio Percoco
pgpizJtO1l_p0.pgp
Description: PGP signature
