On 17 May 2011 22:42, Robert Collins <robe...@robertcollins.net> wrote:
You're probably right to split the internal from the external API. Much as I'd like to see the external one improved, you are right that the internal one is more important right now. But I can't quite get away from thinking about how they ought to relate. :) Tangentially to my post, I saw a demo of Orchestra, and despite that it is only alpha-level it looks quite impressive, and could be a good way to spin up interconnected vms. > > * The lp api is really good in some ways (mostly, coverage) but > > limited in others, and people speak of lazr.restful and co as being > > hard to change or to represent particular things. So it seems like > > this project could either snag on that or be a powerful motivator to > > improve it. > > Indeed. I think we have different constraints and requirements for > internal apis vs external. We don't need long term support (we control > the clients), we don't want totally different dialects when things > change (we want different calls to be isolated so we can be upgrading > individual points not everything in lock-step), we simply can't use > launchpadlib (as I describe in the wiki page), and we need to make > creating and destroying services as lightweight as possible, we want > concurrent requests, and requests to different services; we need our > own business logic (so want data objects not mapped objects). In the > Template/API service I suggest we could split the public API out to > the front end servers; what we use internally is then totally > unrelated (but it may still be RESTful). I agree they are different, though many of those points apply to both the internal webservices approach and external clients: * the things that would make launchpadlib inadequate for Launchpad's own front end also make it inadequate for writing interesting external clients (with concurrency and talking to multiple endpoints being particular cases); * people want to get higher level results than quasi-db-rows per round trip * I'm not sure quite what "make creating and destroying services as lightweight as possible" means but it certainly sounds like I want it for external clients too ;-) * some clients may well be web apps * merely having an unstable API does not seem like a strong reason not to publish it: people would rather have an unstable way to access something than nothing at all (or more likely, screenscraping.) A lot of my questions, and your responses, are about this being a different problem from the existing external API, and that makes sense. I suppose my question then would be: could it be possible to provide an external authenticated version of the internal transport, or some subsection of it, or if not, why? As a story: I'd like it to be possible to prototype a new feature in Launchpad by running the front end for it on an external server. If it requires some back end changes I'd like to put up a branch for Launchpad that adds this, and then to keep doing the frontend development externally. If it turns out well, the app could eventually become indistinguishable from Launchpad proper. I wonder how this could work. > Transactions are slow. GPG is slow. Consistency checks are slow. > Initialisation of data on disk is slow. > > -> fakes from day one > > This is something I wish we had done in bzr much earlier - it would > have informed our design for things like tree transform (which doesn't > run on a transport) by making us measure the cost of the abstraction > vs the cost of using the full implementation in all our other tests. > (Tree transform is nice - this isn't a slight, merely an example I > knew of offhand). I don't draw that conclusion from our experience with bzr: good abstractions and good abstraction-aware tests are great, but fakes should be used when there is a good reason (no sooner or later). Tree transform is actually an interesting example, because there were three possibilities: 0 - what actually happened: it's directly bound to local disk: this is asserted to be the fastest approach, but also inflexible, and we don't actually *know* whether it's any faster than going through an abstraction 1- go through the Transport abstraction, to local disk: then later on you can play tricks with simulating a FAT or NFS disk or just use a fast in-memory transport; I hope to get bzr to this level 2- go through a Transport abstraction to a MemoryTransport fake I think the important thing is to start out at stage 1; you can always go to stage 2 later. But it may turn out that you soon throw away or change the underlying abstraction, and then having a fake of it was waste. Running against a fake has dangers too: it may not match; there can be bugs that occur only when integrated; the fake can skew from the other implementations; and I think some of these ideas can perversely lead to tests that know too much about the code under test. (But this is really a side point; if people add fakes for everything on the first day I will be very curious whether it really turns out well.) > > * So a specific step towards this could be using loggerhead's json > > api to display some branch content inline in the main application? By > > talking to it from the client, or by having the webapp templates proxy > > it? > > Right. If we want webapp clients to view it we either need xhr > permission glue, or to map loggerhead into the lp namespace using > apache's url routing facilities. > If we do it ourselves in the template layer, we don't need either of > those things but we will be dependent on its reponsiveness to meet our > page render times. > > We may want to do both depending on the situation. I'd certainly start > by sending clients to it (e.g. map it into the main namespace via > apache). The Bazaar team is coming to the Launchpad Thunderdome (whatever that is) and I'd like us to do something aligned with the goals of the sprint and also connecting lp and bzr. Perhaps adding better apis to loggerhead and using them from lp would be a good specific topic. loggerhead seems like an example where the same apis could be useful internally and externally. But perhaps not everything is in both categories. > Having got a handle on the major performance > issues that were driving our users batty (so that now they are asking > about features more than speed) I think our next major single-minded > theme needs to be internal development efficiency, which this proposal > is all about (even though it seems to be just architecture :)). <hearty applause> Martin _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : launchpad-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
