On Mon, 2012-05-21 at 15:39 -0600, Deepak Giridharagopal wrote: > On Mon, May 21, 2012 at 2:04 PM, Marc Zampetti > <marc.zampe...@gmail.com> wrote:
> Why wouldn't a DB-agnostic model be used? > > > The short answer is performance. To effectively > implement things we've > got on our roadmap, we need things that (current) > MySQL doesn't > support: array types are critical for efficiently > supporting things > like parameter values, recursive query support is > critical for fast > graph traversal operations, things like INTERSECT are > handy for query > generation, and we rely on fast joins (MySQL's nested > loop joins don't > always cut it). It's much easier for us to support > databases with > these features than those that don't. For fairly > divergent database > targets, it becomes really hard to get the performance > we want while > simultaneously keeping our codebase manageable. > > > I understand the need to not support everything. Having > designed a number of systems that require some of the features > you say you need, I can say with confidence that most of those > issues can be handled without having an RDBMS that has all > those advanced features. So I will respectfully disagree that > you need features you listed. Yes, you may not be able to use > something like ActiveRecord or Hibernate, and have to > hand-code your SQL more often, but there are a number of > techniques that can be used to at least achieve similar > performance characteristics. I think it is a bit dangerous to > assume that your user base can easily and quickly switch out > their RDBMS systems as easy as this announcement seems to > suggest. I'm happy to be wrong if the overall community thinks > that is true, but for something that is as core to one's > infrastructure as Puppet, making such a big change seems > concerning. > > > We aren't using ActiveRecord or Hibernate, and we are using hand-coded > SQL where necessary to wring maximum speed out of the underlying data > store. I'm happy to go into much greater detail about why the features > I listed are important, but I think that's better suited to puppet-dev > than puppet-users. We certainly didn't make this decision cavalierly; > it was made after around a month of benchmarking various solutions > ranging from traditional databases like PostgreSQL to document stores > like MongoDB to KV stores such as Riak to graph databases like Neo4J. > For Puppet's particular type of workload, with Puppet's volume of > data, with Puppet's required durability and safety requirements...I > maintain this was the best choice. > > While I don't doubt that given a large enough amount of time and > enough engineers we could get PuppetDB working fast enough on > arbitrary backing stores (MySQL included), we have limited time and > resources. From a pragmatic standpoint, we felt that supporting a > database that was available on all platforms Puppet supports, that > costs nothing, that has plenty of modules on the Puppet Forge to help > set it up, that has a great reliability record, that meets our > performance needs, and that in the worst case has free/cheap hosted > offerings (such as Heroku) was a reasonable compromise. I didn't had a look to the code itself, but is the postgresql code isolated in its own module? If yes, then that'd definitely help if someone (not saying I'm volunteering :) wants to port the code to MySQL. On a side note, that'd be terriffic Deepak if you would start a thread on the puppet-dev explaining how the postgresql storage has been done to achieve the speed :) -- Brice Figureau Follow the latest Puppet Community evolutions on www.planetpuppet.org! -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
