>>> we store everything as primitives: floats, time, integer, etc... since >>> we need to query on attributes. it seems like versionedobjects might not >>> be useful to our db configuration currently. >> I don't think the former determines the latter -- we have lots of things >> stored as rows of column primitives and query them out as objects, but >> then you're not storing the object and version (unless you do it >> separately) So, if it doesn't buy you anything, then there's no reason >> to use it.
> sorry, i misunderstood this. i thought you were saying ovo may not fit > into Ceilometer. Nope, what I meant was: there's no reason to use the technique of storing serialized objects as blobs in the database if you don't want to store things like that. > i guess to give it more of a real context for us to understand, > regarding the database layer, if we have an events model which consists of: > > - id: uuid > - event_type: string > - generated: timestamp > - raw: dictionary value (not meant for querying, just for auditing > purposes) > - traits: [list of tuples (key, value, type)] > > given this model, each of our backend drivers take this data and using > it's connection to db, stores data accordingly: > - in mongodb, the attributes are all stored in documents similar to > json, raw attr is stored as json Right, so you could store the serialized version of the object in mongo like this very easily. When you go to pull data out of the database later, you have a strict format, and a version tied to it so that you know exactly how it was stored. If you have storage drivers that handle taking the generic thing and turning it into something appropriate for a given store, then it's entirely possible that you are best suited to be tolerant of old data there. In Nova, we treat the object schema as the interface the rest of the code uses and expects. Tolerance of the actual persistence schema moving underneath and over time is hidden in this layer so that things above don't have to know about it. > - in sql, the data is mapped to an Event table, traits are mapped to > different traits tables depending on type, raw attribute stored as a > string. Yep, so when storing in a SQL database, you'd (presumably) not store the serialized blobs, and rather pick the object apart to store it as a row (like most of the things in nova are stored). > considering everything is stored differently depending on db, how does > ovo work? is it normalising it into a specific format pre-storage? how > does different data will different schemas co-exists on the same db? This is completely up to your implementation. You could end up with a top-level object like Event that doesn't implement .save(), and then subclasses like SQLEvent and MongoEvent that do. All the structure could be defined at the top, but the implementations of how to store/retrieve them are separate. The mongo one might be very simple because it can just use the object infrastructure to get the serialized blob and store it. The SQL one would turn the object's fields into an INSERT statement (or a SQLAlchemy thing). > - is there a some version tag applied to each item and a version schema > table created somewhere? The object defines the schema as a list of tightly typed fields, a bunch of methods, and a version. In this purely DB-specific case, all it does is provide you a facade with which to hide things like storing to a different version or format of schema. For projects that send things over RPC and then dump them in the database, it's super convenient that this is all one thing. > - do we need to migrate the db to some handle different set of > attributes and what happens for nosql dbs? No, Nova made no schema changes as a result of moving to objects. > also, from api querying pov, if i want to query a db, how do you > query/filter across different versions? > - does ovo tell the api what versions exists in db and then you can > filter across any attribute from any schema version? Nope, o.vo doesn't do any of this for you magically. It merely sets up a place for you to do that work. In nova, we use them for RPC and DB storage, which means if we have an old node that receives a new object over RPC (or the opposite) we have rules that define how we handle that. Thus, we can apply the same rules to reading the DB, where some objects might be older or newer. > apologies on not understanding how it all works or if the above has > nothing to do with ovo (i wasn't joking about the 'explain it to me like > i'm 5' request :-) ) ... i think part of the wariness is that the code > seemingly does nothing now (or the logic is extremely hidden) but if we > merge these x hundred/thousand lines of code, it will do something later > if something changes. It really isn't magic and really doesn't do a huge amount of work for you. It's a pattern as much as anything, and most of the benefit comes from the serialization and version handling of things over RPC. Part of the reason why my previous responses are so vague is that I really don't care if you use o.vo or not. What I do care about is that critical openstack projects move (quickly) to supporting rolling upgrades, the likes of what nova supports now and the goals we're trying to achieve. If the pattern that nova defined and spun out into the library helps, then that's good for repeatability. However, the model nova chose clearly mostly applies to projects spawned from nova or those that were more or less desgined in its image. I think the goal should be "Monitoring of your cloud doesn't ever have to be turned off to upgrade it." Presumably you never want to leave your cloud unmonitored while you take a big upgrade. How that goal is realized really doesn't matter to me at all, as long we we get there. --Dan __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
