How about reusing Openstack Congress which is already out there and also implements something like nlog?
-- Saurabh (सौरभ) On Mon, Feb 1, 2016 at 11:12 AM, Ben Pfaff <b...@ovn.org> wrote: > On Mon, Feb 01, 2016 at 10:12:49AM -0500, Russell Bryant wrote: > > On 02/01/2016 05:03 AM, Yusheng Wang wrote: > > > We are trying to propose a new computation model for the ovn-northd > > > application. Currently, ovn-northd computes all logical data path > > > flows for every change. This full computation model is simple and > > > helped verifying the concept of logical data path design, but it also > > > suffers performance drawback. > > > > > > In order to move from full computation to incremental computation, > > > more sophisticated tool is needed to manage the complexity associated > > > with both generating more and more complex logical data path flows > > > and dealing with irrelevant sequence of incremental changes. > > > > > > Now we think it is good time to introduce the new computation model - > > > nlog. Nlog is a specialized declarative programming language based on > > > datalog. > > > > > > The nlog computation model has been proved to work well for quite a > > > few systems. We have done a POC in which a brand new nlog compiler > > > and nlog run-time engine are implemented. Nlog program has been > > > written to implement basic ovn features to verify its capabilities. > > > > > > We see a good match of the ovn data schema with nlog computation > > > model. Both ovn-northd and ovn-controller use ovsdb and the data to > > > be processed are tuples. This is also true for nlog programs and > > > minimum data transformation is needed to adapt to the new computation > > > model. > > > > > > This move will take multiple phases and current ovn-northd > > > functionality will not be impacted. The north and south side database > > > schema will not be changed during and after this move. > > > > Thanks for sharing your ideas! > > > > I haven't heard of nlog before. You mentioned that it has worked well > > for quite a few systems. Can you provide some references for where I > > can go to learn more? > > I can provide some information here, because I've been talking to > Yusheng and encouraging him to start a conversation here. Yusheng, by > the way, is a VMware developer working from the Beijing office, who has > been interested in getting involved with OVN for a while and sees an > opportunity here. Hi Yusheng! > > First, nlog has nothing to do with any of at least the first page of > hits on Google, which appear to all be about some kind of logging system > for .NET. > > Instead, this "nlog" is a variant of the Datalog database language. > Datalog is a language with a pretty long pedigree dating back to the > 1980s, at least, but there aren't very many good resources for it > online. Wikipedia has a very basic introduction. A paper I've found > that gives some insight on how to implement it is "What You Always > Wanted to Know About Datalog (And Never Dared to Ask)" here: > > http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.384.5801&rep=rep1&type=pdf > > Personally, I think of nlog as analogous to Makefiles, in that you > specify dependencies and formulas and there's an engine that does all > the smart bits of figuring out what's the minimum amount of computation > to get the outputs you want. > > Yusheng made a prototype implementation of nlog in Java and did a demo > of it for me. I found this pretty impressive. It only took a few > thousand lines of code and yielded something that produces incremental > results for incremental changes to the input data. I think we'll want > something like that for scale at ovn-northd, at least eventually. > (ovn-controller will also want incremental computation but I don't know > whether it should be the same solution.) > > I don't know whether there are public implementations of nlog. The best > reference I know for use in a product is the 2014 NSDI paper on NVP, > especially section 4.3: http://benpfaff.org/papers/net-virt.pdf > > > In general, I definitely agree that this problem is something we need to > > work on. I've been wanting to look into adding a way for users of the > > IDL API to receive callbacks on row events (create, update, delete) so > > that it knows which rows it should apply updates for. That's about as > > far as I've gone though. I haven't written any code yet because I keep > > staying busy with other things. > > That's a necessary feature to implement nlog in ovn-northd. It's there > now, by the way, contributed by HP. > _______________________________________________ > dev mailing list > dev@openvswitch.org > http://openvswitch.org/mailman/listinfo/dev > _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
