The email thread below is an one that I have been having with Luke and some others at IBM who have been working on Trove.
I feel that these kind(s) of conversations are better had in the mailing list, and in replying to this email, I let the participants know that absent any objections, I would like to continue this conversation on the ML. Hearing nothing in the past two days, I am forwarding this to the ML. -amrith > On Thu, Aug 17, 2017 at 4:24 PM, Luke Browning <lukebrown...@us.ibm.com> > wrote: > >> Hi Amrith, >> >> I understand that the Trove project will be going into maintenance mode >> soon. I have a number of Trove related enhancements and bug fixes that I >> would like to submit to the community, but I don't know if they would be >> considered given that the project is going into maintenance mode. Please >> elaborate on what you mean by maintenance mode. >> > > See: https://review.openstack.org/#/c/488947/ > > It is not a foregone conclusion that the project will go into maintenance > mode. Thierry and I (for example) are not convinced that this is the right > course of action, but if there are no offers to contribute to the project, > it may be a decision by default. > > Do you subscribe to the OpenStack-dev mailing list. Please also see > http://openstack.markmail.org/thread/4p6gp263fei4mbru > > Finally, for a description of what maintenance mode means, please refer: > https://governance.openstack.org/tc/reference/tags/status_ma > intenance-mode.html > >> >> >> Would Trove be updated to support new OpenStack versions? >> > Would support be provided for Trove gate testing? >> > Would elements be enhanced to support Xenial and newer versions of >> databases? >> > Is there a time limit associated with maintenance mode? For example, would >> it remain in effect for a year or two after the new stuff is released? >> >> For the four questions above, I direct your attention to the definition > of the maintenance-mode tag (https://governance.openstack. > org/tc/reference/tags/status_maintenance-mode.html) and remind you that > support for different versions from the community, for gate testing and for > elements is dependent on people participating (and their employers allowing > them to do this). At this point, I have no one who has stepped up to do > this in any significant way. There are a couple of people from China Mobile > who want to help but who are largely in the weeds, trying to fix this and > that bug but have no idea what Trove is all about. IBM has a core reviewer > on the project (Mariam is copied on this email). I am happy that she's > able to devote what time she can to the project but absent competent > reviewers, whether your changes ever get merged or not remains an > interesting question. Since you will be contributing elements and propose > to contribute a tool, will you be (or will IBM be) offering to support it? > > Let me provide some back ground on my code changes, so you will have a > better idea of what might be submitted in a patch. > >> >> I have written a fully automated command that creates a virtual guest >> image based on Ubuntu 16.04 containing a user specified version of a >> database. The user's selection is validated against an internal list of >> databases built into the tool. Once validated, the tool creates the image, >> registers it with Glance, and then creates a Trove Datastore definition for >> it. This is done in a single command invocation that typically takes about >> 25 minutes to complete. For some of the databases, it takes considerably >> longer (2 hours for mongodb) as I have to compile source code. The guest >> image that is produced is complete from a Trove perspective. It includes >> the Cloud specific trove-guestagent.conf file and SSH public keys for the >> DBA and OpenStack controller nodes. >> >> What mechanism does it use to develop the image? is it diskimage-builder > or some other? > > > Is there something the matter with the existing tool that exists, it > already works, it does more than just build an image, and it is already > integrated in the gate. Why not use that? > > > This approach is nice (having a full guest image with the guest agent and > all that but as you observe, it takes about 2 hours per build and from a > developers perspective, this is a pain. > > I assume that the tool is therefore for production use cases. > > Could the elements that you have developed be used with the existing tool > or is your tool a complete replacement for the one that already exists? > > Is there some reason that this tool was not developed in the open, using > the normal open development process? > > There are ways to accelerate the compiling of source code and such through > the use of image layers (if you are using DIB). I have elements and code > that will build most of the trove elements in minutes and I'm waiting to > see whether there is any point in contributing them to the upstream or not, > at this point. I could show you how to do that, and it would be awesome to > have you build your tool into the build process somehow. That would be a > win-win for all ... > > I also have commands that setup Nova flavors on a per database basis. >> These flavors can be customized by the user before they are instantiated in >> the cloud. >> > The existing command (trovestack) already does this. > > >> >> I wouldn't expect to drop these commands to the Trove community. Only the >> changes to the Trove-guestagent and elements enabling databases. >> >> Here's the list of databases that are currently supported for ppc64le. In >> some cases, I have had to add Xenial support. >> >> mariadb 10.1 --- package comes from the community >> mongodb 3.4 community edition --- source code from the community is >> compiled as there is no ppc64le package >> mongodb 3.4 enterprise edition --- package comes from community >> mysql 5.7 --- package comes from Ubuntu 16.04. percona-xtrabackup-24 is >> compiled as there is no power support in community >> postgresql 9.5 --- package comes from Ubuntu 16.04 >> postgresql 9.6 --- package comes from community >> redis 3.0 --- package comes from Ubuntu 16.04 >> redis 3.2 --- source code from the community is compiled >> > Why not just contribute the elements to the existing project and have the > current tool use them? > > What of other databases currently supported? galera cluster (pxc), > vertica, db2, percona, couchdb, and couchbase? > > >> >> My element related code changes add environment variables that control >> the database version, the source of the database software (distro, >> community, enterprise), and logging options. In this way, the logic in the >> element is more generalized and can be made to support multiple versions. I >> also added an environment variable that specifies the name of the source >> package to be downloaded and compiled when that is applicable. I also have >> environment variables to enable database and trove guest agent logging. >> > Which is nice except that your guest agent code may or may not know how > to deal with that. > > >> >> >> My trove-guestagent code changes are mostly about not reconfiguring the >> database with configuration data remotely passed to it from the task >> manager. My code changes are isolated to the prepare step. This logic looks >> something like: if database version x, add or remove configuration >> parameter. Note there is relatively very little of this code, ~150 LOCs, as >> database configuration parameters rarely change. Note there is already some >> of this DB version specific logic in the trove guest agent, so I am just >> adding more of it. But I believe a better way is not to alter the database >> configuration data when the database is fully configured in a Trove >> compliant manner via elements. This would be implemented through a new >> command argument to the Trove Guest agent which says ignore configuration >> changes. It would be set by the elements as well. >> > Hard to say whether this is a good approach or not with out more detail. > > >> >> >> I haven't looked into the scale out configuration data yet, but this can >> be handled differently as required. >> >> Not sure what you mean. > > > > >> So, are these the sorts of changes that I could add in a maintenance >> release? >> >> Maintenance release of what, trove? Are you asking whether these changes > can be made in a stable branch? if yes, which branch? See > https://docs.openstack.org/project-team-guide/stable-branches.html and > the section "Appropriate Fixes". > > >> 1) conditionally disable the dynamic installation and configuration of >> database software in the trove-guest agent via a new command argument >> 2) add environment variables to trove elements enabling multiple versions >> of databases to be installed and configured >> 3) add environment variables to trove elements to enable database and >> trove-guest agent logging >> 4) add xenial support to elements >> 5) compiling source code in elements for missing stack elements for >> ppc64le >> >> These changes are implemented at: https://github.com/open-power- >> ref-design-toolkit/os-services/tree/master/osa/dbaas/dbimage-builder >> > OK, so this is a whole new tool, sure you can propose it but how do we > make sure that it gets integrated into the current testing/gate? If you > want to drop this then I'd also have to assume that there would be some > code that would make it part of the current gate, and ideally make it > generate the images for the current gate. But for that, the fact that you > install the guest agent onto the image is a serious drag. > > > Also, why this wasn't proposed as a blueprint or spec in the upstream so > it could be developed in the open, I don't quite understand. After all, you > are now showing up with a complete tool and asking that it be included > which largely runs counter to the whole notion of open development, at > least as I understand it. > > > Finally, once committed, do you (or IBM) commit to maintaining this code > and fixing bugs in it should they arise, and upgrade/update it from time to > time? > > >> >> Thanks, Luke Browning >> >> >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
