-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 All this is great work! Thanks for the write-up as well.
I think I discovered an issue with it - take a look at this bug http://pad.lv/1403738. It seems machine.SetAgentVersion() should be handled specially by the multi-env transaction runner, as only after calling it the upgrade actually starts and the steps to add env-uuids to state collections are executed. On 18.12.2014 05:28, Menno Smits wrote: > I've landed several big changes recently which automate handling > of multi-environment concerns when accessing Juju's collections. > These both simplify DB queries and updates as well as reducing the > risk of unintended data leakage between environments. Although in > most cases you won't even know that anything has changed, it's > worth understanding what's been done. > > *Collections* > > MongoDB queries against collections which contain data for > multiple environments are now automatically modified to ensure they > return records for only the environment tied to State being queried > against. Queries against collections which do not contain data for > multiple environments pass through untouched. > > Some examples ... > > machines.FindId("2") becomes machines.FindId("<uuid>:2").One(&doc) > > machines.Find(bson.D{{"series", "trusty"}}} becomes > machines.Find(bson.D{{"series", "trusty"}, {"env-uuid", "<uuid>"}}) > > > machines.Find(bson.D{{"_id", "4"}}} becomes > machines.Find(bson.D{{"_id", "<uuid>:4"}, {"env-uuid", "<uuid>"}}) > > > Where "<uuid>" is the environment UUID of the State instance the > collection was obtained from (using getCollection()). > > The Remove, RemoveId and RemoveAll methods on collections also > have similar handling and the collection Count method returns only > the number of records in the collection for a single environment. > > The main benefit of this is that you don't need to remember to wrap > ids in State.docID() calls or remember to add the "env-uuid" field > to queries. In fact, I recommend you leave them out to reduce noise > from code that does DB queries. > > There are some limited cases where you might really need to query > across multiple environments or don't want the automatic munging in > place for some reason. For these scenarios you can get hold of a > *mgo.Collection by calling State.getRawCollection(). This is > currently only being used by a few database migration steps. > > Note that query selectors using MongoDB operators with the _id > field will be left untouched. In these cases you need to know that > there's a UUID prefix on the _id and handle it yourself. For > example, to query all the machines with ids starting with "4" you > might consider doing: > > machines.Find(bson.D{{"_id", bson.D{"$regex": "^4.*"}}}} which is > transformed to: machines.Find(bson.D{{"_id", bson.D{"$regex": > "^4.*"}}}}, {"env-uuid", "<uuid>"}}) > > Note how the _id selector is left alone but the env-uuid selector > is still added. It's left up to the developer to account for the > environment UUID in _id regex (the regex above won't work as is). > > > *Transactions* > > Changes have also been made for automatically modifying > transaction operations to account for multi-environment > collections. > > For example: > > st.runTransaction([]txn.Op{{ C: machinesC, Id: "1" Remove: true, }, > { C: machinesC, Id: "2", Insert: bson.D{ {"series", "trusty"}, }, > }, { C: machinesC, Id: "3", Insert: &machineDoc{ Series: "trusty", > }, }, { C: otherC, Id: "foo", Insert: bson.D{}, }}) > > automatically becomes: > > st.runTransaction([]txn.Op{{ C: machinesC, Id: "<uuid>:1", Remove: > true, }, { C: machinesC, Id: "<uuid>:2", Insert: bson.D{ {"_id", > "<uuid>:2"}, {"env-uuid", "<uuid>"}, {"series", "trusty"}, }, }, { > C: machinesC, Id: "<uuid>:3", Insert: &machineDoc{ DocID: > "<uuid>:3", EnvUUID: "<uuid>", Series: "trusty", } }, { C: otherC, > Id: "foo", Insert: bson.D{}, }}) > > Note how the environment UUID is prefixed onto ids for operations > for multi-environment collections. Also see how the _id and > env-uuid field on documents defined using bson.D or structs (bson.M > supported too) are automatically populated. A panic will occur if > you provide the environment UUID but it doesn't match what was > expected as this indicates a likely bug. > > Any document updates are made in place so that the caller sees them > once the transaction completes. This makes it safe for the caller > to a document struct used with a transaction operation for further > work - the struct will match what was written to the DB. Note that > if a struct is passed by value and needs updating, a panic will > occur. This won't normally be a problem as we tend to use pointers > to document structs with transaction operations, and the panic is a > helpful indication that the document provided isn't > multi-environment safe. > > Note that only the Id and Insert fields of txn.Op are touched. The > Update and Assert fields are left alone. > > In some cases you may need to run a transaction without invoking > automatic multi-environment munging. State now has a rawTxnRunner() > and runRawTransaction() methods for the rare situations where this > is necessary. Please use these sparingly. > > *Performance* * * With the extra work being done to implement > automatic multi-environment support, the performance impact was a > concern. I have compared multiple runs of the state package unit > tests, with and without these changes and the difference is lost in > the noise. > > If you see any problems or have any questions, please let me know. > > - Menno > > - -- Dimiter Naydenov <[email protected]> juju-core team -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iQEcBAEBAgAGBQJUkwixAAoJENzxV2TbLzHwgCEIAMihgys62djSGdmaVYTgsYVM YoJ3SNXZlfn6X5hBz9RrYtfC0K3vkW1t8jJMQtaShwH9pakMmIeUQXBWTNIZUqmf WQyboYW65RtnjWfR/oSQKHXr02wz7Trxeh0oM9TqINPEnllq5jubHrYscmCkZfQj BmaAXTNozYm/KKDN6ros0AbCWZdd1nn/PvG38XbMtBw1N/wK0qAZV92nl8DW+o3j coaoQ9KdGdhAdnMcub4u6UVMUp4V/2IiiPE379ATn8Yo6qdysd5yEnqWad0ajPnV WFsfjqZrY1qc9IxqaFjfRhszkaQgM7392dEejW+TVU69qzvlrUZHIQLv9hl52Ss= =ugvU -----END PGP SIGNATURE----- -- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
