On Mon, Feb 01, 2016 at 10:12:49AM -0500, Russell Bryant wrote: > On 02/01/2016 05:03 AM, Yusheng Wang wrote: > > We are trying to propose a new computation model for the ovn-northd > > application. Currently, ovn-northd computes all logical data path > > flows for every change. This full computation model is simple and > > helped verifying the concept of logical data path design, but it also > > suffers performance drawback. > > > > In order to move from full computation to incremental computation, > > more sophisticated tool is needed to manage the complexity associated > > with both generating more and more complex logical data path flows > > and dealing with irrelevant sequence of incremental changes. > > > > Now we think it is good time to introduce the new computation model - > > nlog. Nlog is a specialized declarative programming language based on > > datalog. > > > > The nlog computation model has been proved to work well for quite a > > few systems. We have done a POC in which a brand new nlog compiler > > and nlog run-time engine are implemented. Nlog program has been > > written to implement basic ovn features to verify its capabilities. > > > > We see a good match of the ovn data schema with nlog computation > > model. Both ovn-northd and ovn-controller use ovsdb and the data to > > be processed are tuples. This is also true for nlog programs and > > minimum data transformation is needed to adapt to the new computation > > model. > > > > This move will take multiple phases and current ovn-northd > > functionality will not be impacted. The north and south side database > > schema will not be changed during and after this move. > > Thanks for sharing your ideas! > > I haven't heard of nlog before. You mentioned that it has worked well > for quite a few systems. Can you provide some references for where I > can go to learn more?
I can provide some information here, because I've been talking to Yusheng and encouraging him to start a conversation here. Yusheng, by the way, is a VMware developer working from the Beijing office, who has been interested in getting involved with OVN for a while and sees an opportunity here. Hi Yusheng! First, nlog has nothing to do with any of at least the first page of hits on Google, which appear to all be about some kind of logging system for .NET. Instead, this "nlog" is a variant of the Datalog database language. Datalog is a language with a pretty long pedigree dating back to the 1980s, at least, but there aren't very many good resources for it online. Wikipedia has a very basic introduction. A paper I've found that gives some insight on how to implement it is "What You Always Wanted to Know About Datalog (And Never Dared to Ask)" here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.384.5801&rep=rep1&type=pdf Personally, I think of nlog as analogous to Makefiles, in that you specify dependencies and formulas and there's an engine that does all the smart bits of figuring out what's the minimum amount of computation to get the outputs you want. Yusheng made a prototype implementation of nlog in Java and did a demo of it for me. I found this pretty impressive. It only took a few thousand lines of code and yielded something that produces incremental results for incremental changes to the input data. I think we'll want something like that for scale at ovn-northd, at least eventually. (ovn-controller will also want incremental computation but I don't know whether it should be the same solution.) I don't know whether there are public implementations of nlog. The best reference I know for use in a product is the 2014 NSDI paper on NVP, especially section 4.3: http://benpfaff.org/papers/net-virt.pdf > In general, I definitely agree that this problem is something we need to > work on. I've been wanting to look into adding a way for users of the > IDL API to receive callbacks on row events (create, update, delete) so > that it knows which rows it should apply updates for. That's about as > far as I've gone though. I haven't written any code yet because I keep > staying busy with other things. That's a necessary feature to implement nlog in ovn-northd. It's there now, by the way, contributed by HP. _______________________________________________ dev mailing list dev@openvswitch.org http://openvswitch.org/mailman/listinfo/dev
