Andy Zhou spent some time last week investigating etcd 3.0 as a possible
database target for OVN. Based on some of what he found out, and a
little digging of my own, here's a summary of my thoughts on it in the
format used previously for
http://openvswitch.org/pipermail/dev/2016-March/067479.html:
- etcd 3.0. This is in beta at time of this writing. Much of the
information here is gleaned from
https://github.com/coreos/etcd/releases/tag/v3.0.0-beta.0 and
https://github.com/coreos/etcd/blob/master/ROADMAP.md. The
latter is out of date: it says that 3.0 would be released in
April and it is now June.
etcd 3.0 allows clients to receive a notification when a key's
value changes. Notification can occur for specific keys or for
all keys with a given bytewise prefix, which would allow
monitoring of entire OVSDB tables. A testing report from Andy
Zhou indicates that the number of these registrations is limited
to about 800, although it wasn't clear to him whether that was a
limit or a bug. If this is a limit, it would disqualify etcd
because as a bare minimum each of 1000+ hypervisors would need
to register at least one prefix (and probably more).
etcd 3.0 implements transactions. A testing report from Andy
Zhou indicates that the size of a transaction is limited to
about 100 keys, although it wasn't clear to him whether that was
a limit or a bug. If this is a limit, it would disqualify etcd
unless we layered some kind of higher-level transaction system
on top of it or dropped the need for transactions.
etcd 3.0 only has a Go client library. The roadmap for etcd 3.1
says that it will support more languages, but C support is
unlikely at that time (Java and C++ seem more probable). We
will probably have to build our own client library. The etcd
2.x and 3.x protocols are completely different, so we could not
start from a C client for 2.x. The etcd 3.x protocol is based
on gRPC, so we would probably want to use the official gRPC
library for C available at https://github.com/grpc/grpc.
etcd officially supports only the x86-64 architecture. It has
known bugs on 32-bit architectures. Its portability bugs are
serious enough that by default it aborts on startup if run on
anything other than x86-64. This would cause problems for
testing OVN on anything other than x86-64, but it might not be a
serious issue for OVN deployments since presumably a few x86-64
boxes could be deployed alongside whatever other architectures
were in use.
etcd has official binaries for GNU/Linux, Mac OS, and Windows,
and etcd 2.x is in the FreeBSD ports tree, so OS support does
not seem to be an issue.
This leaves me with the following discussion questions:
- Is lack of non-amd64 support a blocker? It's going to suck for
me, because I do my development on i386 specifically to introduce
diversity (I find bugs that others don't because they're all on
amd64). Also, we've lately had people actually making sure that
the testsuite passes on non-x86-64, so there seems to be real
interest there.
- Can etcd handle big transactions?
- Can etcd handle a sufficient number of notify registrations?
Thanks,
Ben.
_______________________________________________
discuss mailing list
discuss@openvswitch.org
http://openvswitch.org/mailman/listinfo/discuss
