Andy Zhou spent some time last week investigating etcd 3.0 as a possible database target for OVN. Based on some of what he found out, and a little digging of my own, here's a summary of my thoughts on it in the format used previously for http://openvswitch.org/pipermail/dev/2016-March/067479.html:
- etcd 3.0. This is in beta at time of this writing. Much of the information here is gleaned from https://github.com/coreos/etcd/releases/tag/v3.0.0-beta.0 and https://github.com/coreos/etcd/blob/master/ROADMAP.md. The latter is out of date: it says that 3.0 would be released in April and it is now June. etcd 3.0 allows clients to receive a notification when a key's value changes. Notification can occur for specific keys or for all keys with a given bytewise prefix, which would allow monitoring of entire OVSDB tables. A testing report from Andy Zhou indicates that the number of these registrations is limited to about 800, although it wasn't clear to him whether that was a limit or a bug. If this is a limit, it would disqualify etcd because as a bare minimum each of 1000+ hypervisors would need to register at least one prefix (and probably more). etcd 3.0 implements transactions. A testing report from Andy Zhou indicates that the size of a transaction is limited to about 100 keys, although it wasn't clear to him whether that was a limit or a bug. If this is a limit, it would disqualify etcd unless we layered some kind of higher-level transaction system on top of it or dropped the need for transactions. etcd 3.0 only has a Go client library. The roadmap for etcd 3.1 says that it will support more languages, but C support is unlikely at that time (Java and C++ seem more probable). We will probably have to build our own client library. The etcd 2.x and 3.x protocols are completely different, so we could not start from a C client for 2.x. The etcd 3.x protocol is based on gRPC, so we would probably want to use the official gRPC library for C available at https://github.com/grpc/grpc. etcd officially supports only the x86-64 architecture. It has known bugs on 32-bit architectures. Its portability bugs are serious enough that by default it aborts on startup if run on anything other than x86-64. This would cause problems for testing OVN on anything other than x86-64, but it might not be a serious issue for OVN deployments since presumably a few x86-64 boxes could be deployed alongside whatever other architectures were in use. etcd has official binaries for GNU/Linux, Mac OS, and Windows, and etcd 2.x is in the FreeBSD ports tree, so OS support does not seem to be an issue. This leaves me with the following discussion questions: - Is lack of non-amd64 support a blocker? It's going to suck for me, because I do my development on i386 specifically to introduce diversity (I find bugs that others don't because they're all on amd64). Also, we've lately had people actually making sure that the testsuite passes on non-x86-64, so there seems to be real interest there. - Can etcd handle big transactions? - Can etcd handle a sufficient number of notify registrations? Thanks, Ben. _______________________________________________ discuss mailing list discuss@openvswitch.org http://openvswitch.org/mailman/listinfo/discuss