Hi, Openstackers I would like to propose a new initiative to implement AWS DynamoDB API for OpenStack.
I believe this is a right moment for OpenStack community to start MagnetoDB discussion and design around such service as more data processing and management services, such as Savanna and Trove, are appearing in OpenStack, creating a sufficient basis for higher level data APIs like DynamoDB API. With Trove you can provision and manage different databases including NoSQL databases support, which will appear soon. It minimizes the maintenance process and turns database deployment and management to API calls. DynamoDB [1] <http://aws.amazon.com/dynamodb/> is well-known key-value NoSQL storage provided by Amazon with HTTP interface. It uses JSON as its transfer data model. The data is stored as key-value, where value is list of attributes. The attribute’s value can be String, Number, Binary or sets of those types. Read consistency can be strong or eventual per request. The secondary indexes are also supported. Simplicity, reliability, and pleasant documentation have made it actively used in a lot of applications, but hosting them on OpenStack is impossible due to the absence of such storage. DynamoDB is already widely used and has its own niche [2] <http://aws.amazon.com/dynamodb/testimonials/> despite being closed and proprietary. Let me share the idea to implement a similar service which imitates DynamoDB API [3]<http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/API.html>, called MagnetoDB. MagnetoDB will fully implement DynamoDB API thus enabling the possibility for cloud users to migrate their applications that use DynamoDB from AWS to OpenStack. For backend storage provisioning and management, OpenStack DBaaS Trove will be used. The backend database should be pluggable to provide flexibility in terms choosing the solution which best matched the existing or planned OpenStack installation technology stack. MySQL is one of the obvious options, being the de-facto standard for use with OpenStack and being supported by Trove right away. However, users of DynamoDB expect high performance, scalability, and availability. Those can be achieved more easily by using one of the NoSQL solutions in MagnetoDB. Apache Cassandra [4] <http://cassandra.apache.org/>looks very suitable for that case, due to its tunable consistency, easy scalability, key-value data model, ring topology and other features that give us predictable high performance and fault tolerance. The Cassandra data model perfectly fits MagnetoDB needs. Moreover, support of NoSQL databases is in Trove roadmap [5]<https://blueprints.launchpad.net/trove/+spec/cassandra-db-support>and the fist version of Cassandra provisioning is currently in progress [6] <https://review.openstack.org/#/c/51884/>. AWS SDK [7]<http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/UsingAWSSDK.html>can be used to query data. Provided by Amazon, it hides all protocol stuff giving the high level API. In Python world you can use the rather common Boto [8] <https://github.com/boto/boto> [9]<http://boto.readthedocs.org/en/latest/dynamodb2_tut.html>library. An initial draft of the more official proposal is available here: https://wiki.openstack.org/wiki/MagnetoDB Any contribution, comments and pieces of advice are very welcome Also, I plan to make the lightning talk on Tuesday, November 5 at Expo Breakout Room 1. See you on summit. [1] http://aws.amazon.com/dynamodb [2] http://aws.amazon.com/dynamodb/testimonials/ [3] http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/API.html [4] http://cassandra.apache.org/ [5] https://blueprints.launchpad.net/trove/+spec/cassandra-db-support [6] https://review.openstack.org/#/c/51884/ [7] http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/UsingAWSSDK.html [4] https://github.com/boto/boto [5] http://boto.readthedocs.org/en/latest/dynamodb2_tut.html With best regards, Ilya Sviridov <http://www.mirantis.ru/>
_______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
