Yis! <3
On 3 August 2013 21:15, Jan Lehnardt <j...@apache.org> wrote: > More update: > > I started producing a plugin template repo that people can clone to > build their own plugins along with a comprehensive README.md: > > https://github.com/janl/my-first-couchdb-plugin > > The idea is to move the README to the CouchDB docs eventually and ship > the plugin template with CouchDB, so people can get started easily. > > Best > Jan > -- > > On Aug 3, 2013, at 17:53 , Simon Metson <si...@cloudant.com> wrote: > > > :) > > > > > > On Saturday, 3 August 2013 at 14:21, Jan Lehnardt wrote: > > > >> Couldn’t help but implement it. It’s in the branch now. > >> > >> Jan > >> -- > >> > >> On Aug 3, 2013, at 08:12 , Simon Metson <si...@cloudant.com> wrote: > >> > >>> Sounds good to me. > >>> > >>> > >>> On Saturday, 3 August 2013 at 00:56, Jan Lehnardt wrote: > >>> > >>>> > >>>> On Aug 3, 2013, at 00:02 , Russell Branca <chewbra...@apache.org(mailto: > chewbra...@apache.org)> wrote: > >>>> > >>>>> This is fantastic, Jan! Glad to see this coming along. > >>>>> > >>>>> One of the goals with Fauxton has always been to make it easy for > plugins > >>>>> to extend the interface and provide new functionality. I've been > toying > >>>>> with the idea of having a _fauxton db that plugins install to as > docs with > >>>>> attachments, but that's for a different thread. The general idea > here is > >>>>> that a plugin will be able to extend Fauxton by adding a new page > with it's > >>>>> own functionality, or hook into existing pages to extend other areas. > >>>>> > >>>>> For instance, you could have a couchdb-lucene plugin that hooks into > the > >>>>> databases list and allows you to add interfaces for building full > text > >>>>> indexes and searching on existing indexes. Or you could have a > dedicated > >>>>> page for Geocouch, or whatever. > >>>>> > >>>>> The functionality is there, but it's still a bit of a manual > process, so > >>>>> we'll need to make it more dynamic and smooth out the rough edges. > >>>>> > >>>>> I'm very excited to see progress being made on plugins, great work! > >>>> > >>>> Thanks, I’m glad you like this! :) > >>>> > >>>> Another way to get the Fauxton plugin loaded would be to extend the > >>>> /_plugins API endpoint, so Fauxton could request GET > /_plugins/<pluginname>/ > >>>> and it would serve <couchdblibdir>/plugins/<pluginname/priv/www which > is > >>>> just a place for Fauxton-enabled plugins. > >>>> > >>>> Fauxton would walk /_config/plugins/ to get to a list of plugins. > >>>> > >>>> In fact that should be pretty simple to set up. > >>>> > >>>> For now I am trying to avoid having a custom database for this, mostly > >>>> because I don’t think there are many advantages (e.g. replication of > >>>> plugins?) and code complexity. These priorities might change in the > >>>> future, but for now I am happy to get this working at all :) > >>>> > >>>> If you are okay with the above plan of serving plugin HTML/JS/CSS from > >>>> /_plugins/<pluginname>, I’m happy to add this to the branch. > >>>> > >>>> Best > >>>> Jan > >>>> -- > >>>> > >>>> > >>>> > >>>>> > >>>>> > >>>>> -Russell > >>>>> > >>>>> > >>>>> On Fri, Aug 2, 2013 at 2:17 PM, Jan Lehnardt <j...@apache.org(mailto: > j...@apache.org)> wrote: > >>>>> > >>>>>> And a few more (from COUCHDB-1867): > >>>>>> > >>>>>> - Add uninstall, incl. Futon UI. > >>>>>> - Only install a plugin if the source and target CouchDB version > matches. > >>>>>> - Rebase against master. > >>>>>> > >>>>>> * * * > >>>>>> > >>>>>> This concludes my list for a Minimally Viable Plugin feature. (See > the > >>>>>> original email or README.md (http://README.md)* for the roadmap) > >>>>>> > >>>>>> I’d appreciate some more reviews & feedback**, but other than that, > I’d be > >>>>>> happy to ship this as an experimental feature in any next release. > >>>>>> > >>>>>> * > >>>>>> > https://github.com/janl/couchdb/blob/1867-feature-plugins/src/couch_plugins/README.md#roadmap > >>>>>> ** > >>>>>> > https://github.com/janl/couchdb/compare/apache:master...1867-feature-plugins > >>>>>> > >>>>>> > >>>>>> Best > >>>>>> Jan > >>>>>> -- > >>>>>> > >>>>>> On Aug 1, 2013, at 19:34 , Jan Lehnardt <j...@apache.org (mailto: > j...@apache.org)> wrote: > >>>>>> > >>>>>>> A few updates: > >>>>>>> > >>>>>>> By Bob Ippolito / @etrepum: > >>>>>>> - Plugins are now installed in libdir (instead of /tmp). > >>>>>>> - Config loading is now done with proper .ini files. > >>>>>>> - Various cleanups and code review (Thanks!). > >>>>>>> > >>>>>>> Mine (most suggested by Bob): > >>>>>>> - `plugins.html` now shows you if a plugin is already installed. > >>>>>>> and which version, if it doesn’t match the installable one. > >>>>>>> - The Install button now disables after an installation. > >>>>>>> - Plugins are now registered with couch_config as > >>>>>>> /_config/plugins/name = version > >>>>>>> - Updated `couch-config` to print --erlang-version and --erl-bin > >>>>>>> - Updated the geocouch plugin to use the new options in > >>>>>>> `couch-config`. > >>>>>>> - Added Bob Ippolito’s couchperuser plugin to Futon. > >>>>>>> > >>>>>>> > >>>>>>> Best > >>>>>>> Jan > >>>>>>> -- > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Jul 31, 2013, at 19:07 , Jan Lehnardt <j...@apache.org (mailto: > j...@apache.org)> wrote: > >>>>>>> > >>>>>>>> Heya, > >>>>>>>> > >>>>>>>> I couldn’t help myself thinking about plugin stuff and ended up > >>>>>>>> whipping up a proof of concept. > >>>>>>>> > >>>>>>>> Here’s a <1 minute demo video: > >>>>>>>> > >>>>>>>> > https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.mov > >>>>>>>> > >>>>>>>> Alternative encoding: > >>>>>>>> > >>>>>>>> > https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.m4v) > >>>>>>>> > >>>>>>>> > >>>>>>>> In my head the whole plugin idea is a very wide area, but I was so > >>>>>>>> intrigued by the idea of getting something running with a click > on a > >>>>>>>> button in Futon. So I looked for a minimally viable plugin system. > >>>>>>>> > >>>>>>>> > >>>>>>>> ## Design principles > >>>>>>>> > >>>>>>>> It took me a day to put this all together and this was only > possible > >>>>>>>> because I took a lot of shortcuts. I believe they are all viable > for a > >>>>>>>> first iteration of a plugins system: > >>>>>>>> > >>>>>>>> 1. Install with one click on a button in Futon (or HTTP call) > >>>>>>>> 2. Only pure Erlang plugins are allowed. > >>>>>>>> 3. The plugin author must provide a binary package for each > Erlang (and, > >>>>>>>> later, each CouchDB version). > >>>>>>>> 4. Complete trust-based system. You trust me to not do any nasty > things > >>>>>>>> when you click on the install button. No crypto, no nothing. Only > >>>>>>>> people who can commit to Futon can release new versions of > plugins. > >>>>>>>> 5. Minimal user-friendlyness: won’t install plugins that don’t > match > >>>>>>>> the current Erlang version, gives semi-sensible error messages > >>>>>>>> (wrapped in a HTTP 500 response :) > >>>>>>>> 6. Require a pretty strict format for binary releases. > >>>>>>>> > >>>>>>>> > >>>>>>>> ## Roadmap > >>>>>>>> > >>>>>>>> Here’s a list of things this first iterations does and doesn’t do: > >>>>>>>> > >>>>>>>> - Pure Erlang plugins only. No C-dependencies, no JavaScript, no > >>>>>> nothing. > >>>>>>>> - No C-dependencies. > >>>>>>>> - Install a plugin via Futon (or HTTP call). Admin only. > >>>>>>>> - A hardcoded list of plugins in Futon. > >>>>>>>> - Loads a pre-packaged, pre-compiled .tar.gz file from a URL. > >>>>>>>> - Only installs if Erlang version matches. > >>>>>>>> - No security checking of binaries. > >>>>>>>> - No identity checking of binaries. > >>>>>>>> > >>>>>>>> Here are a few things I want to add before I call it MVP*: > >>>>>>>> > >>>>>>>> - Uninstall a plugin via Futon (or HTTP call). Admin only. > >>>>>>>> - Only installs if CouchDB version matches. > >>>>>>>> - Binaries must be published on *.apache.org (http://apache.org). > >>>>>>>> - Register installed plugins in the config system. > >>>>>>>> - Make sure plugins start with the next restart of CouchDB. > >>>>>>>> - Show when a particular plugin is installed. > >>>>>>>> > >>>>>>>> *MVP hopefully means you agree we can ship this with a few > warnings > >>>>>>>> so people can get a hang of it. > >>>>>>>> > >>>>>>>> Here is a rough list of features squared against future > milestones: > >>>>>>>> > >>>>>>>> Milestone 2: Be creator friendly > >>>>>>>> - Make it easy to build a CouchDB plugin by providing one or more > easy > >>>>>>>> to start templates. > >>>>>>>> - Make it easy to publish new plugins and new versions of existing > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> plugins. > >>>>>>>> - Make it easy to supply packages for multiple Erlang & CouchDB > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> versions. > >>>>>>>> > >>>>>>>> Milestone 3: Public registry > >>>>>>>> - Instead of hardcoding a list of plugins into Futon/Fauxton, we > load > >>>>>>>> a list of applicable plugins from a central (and configurable) > >>>>>>>> plugins repository. > >>>>>>>> - This allows plugin authors to publish new plugins and new > versions > >>>>>>>> of existing plugins independently. > >>>>>>>> > >>>>>>>> Milestone 4: Other Languages > >>>>>>>> - Figure out how to handle C-dependencies for Erlang plugins. > >>>>>>>> - Figure out how to allow other language plugins > >>>>>>>> (c.f. non-JS query servers) > >>>>>>>> > >>>>>>>> Milestone X: Later > >>>>>>>> - Add some account/identity/maybe crypto-web-of-trust system for > >>>>>>>> authors to publish “legit” plugins. > >>>>>>>> - Sign & verify individual releases. > >>>>>>>> > >>>>>>>> A few more things that can happen concurrently depending on what > >>>>>>>> plugins require: > >>>>>>>> - Integrate Erlang/JS tests in the installation > >>>>>>>> - Integrate docs > >>>>>>>> > >>>>>>>> > >>>>>>>> ## How it works > >>>>>>>> > >>>>>>>> This plugin system lives in `src/couch_plugins` and is a tiny > CouchDB > >>>>>>>> module. > >>>>>>>> > >>>>>>>> It exposes one new API endpoint `/_plugins` that an admin user can > >>>>>>>> POST to. > >>>>>>>> > >>>>>>>> The additional Futon page lives at /_utils/plugins.html it is > >>>>>>>> hardcoded. > >>>>>>>> > >>>>>>>> Futon (or you) post an object to `/_plugins` with four properties: > >>>>>>>> > >>>>>>>> { > >>>>>>>> "name": "geocouch", // name of the plugin, must be unique > >>>>>>>> "url": "http://people.apache.org/~jan";, // “base URL” for plugin > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> releases (see below) > >>>>>>>> "version": "couchdb1.2.x_v0.3.0-11-gd83ba22", // whatever version > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> internal to the plugin > >>>>>>>> "checksums": { > >>>>>>>> "R15B03": "ZetgdHj2bY2w37buulWVf3USOZs=" // base64’d sha hash > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> over the binary > >>>>>>>> } > >>>>>>>> } > >>>>>>>> > >>>>>>>> `couch_plugins` then attempts to download a .tar.gz from this > >>>>>>>> location: > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > http://people.apache.org/~jan/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03.tar.gz > >>>>>>>> > >>>>>>>> It should be obvious how the URL is constructed from the POST > data. > >>>>>>>> (This url is live, feel free to play around with this tarball). > >>>>>>>> > >>>>>>>> Next it calculates the sha hash for the downloaded .tar.gz file > and > >>>>>>>> matches it against the correct version in the `checksums` > parameter. > >>>>>>>> > >>>>>>>> If that succeeds, we unpack the .tar.gz file (currently in `/tmp`, > >>>>>>>> need to find a better place for this) and adds the extracted > directory > >>>>>>>> to the Erlang code path > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > (`code:add_path("/tmp/couchdb_plugins/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03/ebin")`) > >>>>>>>> and loads the included application (`application:load(geocouch)`). > >>>>>>>> > >>>>>>>> Then it looks into the `./config` directory that lives next to > `ebin/` > >>>>>>>> in the plugin directory for a file `config.erlt` (“erl-terms”). > with a > >>>>>>>> list of configuration parameters to load. We parse the file and > set > >>>>>>>> the config directives one by one. > >>>>>>>> > >>>>>>>> If that all goes to plan, we report success back to the HTTP > caller. > >>>>>>>> > >>>>>>>> That’s it! :) > >>>>>>>> > >>>>>>>> It’s deceptively simple, probably does a few things very wrong and > >>>>>>>> leaves a few things open (see above). > >>>>>>>> > >>>>>>>> One open question I’d like an answer for is finding a good > location to > >>>>>>>> unpack & install the plugin files that isn’t `tmp`. If the answer > is > >>>>>>>> different for a pre-BigCouch/rcouch-merge and > post-BigCouch/rcouch- > >>>>>>>> merge world, I’d love to know :) > >>>>>>>> > >>>>>>>> > >>>>>>>> ## Code > >>>>>>>> > >>>>>>>> The main branch for this is 1867-feature-plugins: > >>>>>>>> > >>>>>>>> ASF: > >>>>>> > https://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=log;h=refs/heads/1867-feature-plugins > >>>>>>>> GitHub: > https://github.com/janl/couchdb/compare/1867-feature-plugins > >>>>>>>> > >>>>>>>> I created a branch on GeoCouch that adds a few lines to its > `Makefile` > >>>>>>>> that shows how a binary package is built: > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> > https://github.com/janl/geocouch/compare/couchbase:couchdb1.3.x...couchdb1.3.x-plugins > >>>>>>>> > >>>>>>>> * * * > >>>>>>>> > >>>>>>>> I hope you like this :) Please comment and improve heavily! > >>>>>>>> > >>>>>>>> Let me know if you have any questions :) > >>>>>>>> > >>>>>>>> If you have any criticism, please phrase it in a way that we can > use > >>>>>>>> to improve this, thanks! > >>>>>>>> > >>>>>>>> Best, > >>>>>>>> Jan > >>>>>>>> -- > >>>>>>>> > >>>>>>> > >>>>>> > >>>>>> > >>>>> > >>>>> > >>>> > >>>> > >>> > >>> > >> > >> > >> > > > > > > -- Noah Slater https://twitter.com/nslater
