On 04.08.2013, at 18:03, Octavian Damiean <mainer...@gmail.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Oh yea, forgot to tell. Done. :) Merged, thanks! > > On 2013-08-04 17:39, Jan Lehnardt wrote: >> >> On Aug 4, 2013, at 17:29 , Octavian Damiean <mainer...@gmail.com> >> wrote: >> >>> Signed PGP part Great work! >>> >>> Got a quick question. In the notice you mention that the >>> instructions work only with the 1867-feature-plugin branch but >>> you didn't add steps to checkout said branch in the preparation >>> step. >>> >>> Want me to add that bit of information or was that deliberate? >> >> Total oversight, you are more then welcome to add this :) >> >> Jan -- >> >> >> >>> >>> Cheers, Octavian >>> >>> On 2013-08-03 22:15, Jan Lehnardt wrote: >>>> More update: >>>> >>>> I started producing a plugin template repo that people can >>>> clone to build their own plugins along with a comprehensive >>>> README.md: >>>> >>>> https://github.com/janl/my-first-couchdb-plugin >>>> >>>> The idea is to move the README to the CouchDB docs eventually >>>> and ship the plugin template with CouchDB, so people can get >>>> started easily. >>>> >>>> Best Jan -- >>>> >>>> On Aug 3, 2013, at 17:53 , Simon Metson <si...@cloudant.com> >>>> wrote: >>>> >>>>> :) >>>>> >>>>> >>>>> On Saturday, 3 August 2013 at 14:21, Jan Lehnardt wrote: >>>>> >>>>>> Couldn’t help but implement it. It’s in the branch now. >>>>>> >>>>>> Jan -- >>>>>> >>>>>> On Aug 3, 2013, at 08:12 , Simon Metson >>>>>> <si...@cloudant.com> wrote: >>>>>> >>>>>>> Sounds good to me. >>>>>>> >>>>>>> >>>>>>> On Saturday, 3 August 2013 at 00:56, Jan Lehnardt wrote: >>>>>>> >>>>>>>> >>>>>>>> On Aug 3, 2013, at 00:02 , Russell Branca >>>>>>>> <chewbra...@apache.org (mailto:chewbra...@apache.org)> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> This is fantastic, Jan! Glad to see this coming >>>>>>>>> along. >>>>>>>>> >>>>>>>>> One of the goals with Fauxton has always been to make >>>>>>>>> it easy for plugins to extend the interface and >>>>>>>>> provide new functionality. I've been toying with the >>>>>>>>> idea of having a _fauxton db that plugins install to >>>>>>>>> as docs with attachments, but that's for a different >>>>>>>>> thread. The general idea here is that a plugin will >>>>>>>>> be able to extend Fauxton by adding a new page with >>>>>>>>> it's own functionality, or hook into existing pages >>>>>>>>> to extend other areas. >>>>>>>>> >>>>>>>>> For instance, you could have a couchdb-lucene plugin >>>>>>>>> that hooks into the databases list and allows you to >>>>>>>>> add interfaces for building full text indexes and >>>>>>>>> searching on existing indexes. Or you could have a >>>>>>>>> dedicated page for Geocouch, or whatever. >>>>>>>>> >>>>>>>>> The functionality is there, but it's still a bit of >>>>>>>>> a manual process, so we'll need to make it more >>>>>>>>> dynamic and smooth out the rough edges. >>>>>>>>> >>>>>>>>> I'm very excited to see progress being made on >>>>>>>>> plugins, great work! >>>>>>>> >>>>>>>> Thanks, I’m glad you like this! :) >>>>>>>> >>>>>>>> Another way to get the Fauxton plugin loaded would be >>>>>>>> to extend the /_plugins API endpoint, so Fauxton could >>>>>>>> request GET /_plugins/<pluginname>/ and it would serve >>>>>>>> <couchdblibdir>/plugins/<pluginname/priv/www which is >>>>>>>> just a place for Fauxton-enabled plugins. >>>>>>>> >>>>>>>> Fauxton would walk /_config/plugins/ to get to a list >>>>>>>> of plugins. >>>>>>>> >>>>>>>> In fact that should be pretty simple to set up. >>>>>>>> >>>>>>>> For now I am trying to avoid having a custom database >>>>>>>> for this, mostly because I don’t think there are many >>>>>>>> advantages (e.g. replication of plugins?) and code >>>>>>>> complexity. These priorities might change in the >>>>>>>> future, but for now I am happy to get this working at >>>>>>>> all :) >>>>>>>> >>>>>>>> If you are okay with the above plan of serving plugin >>>>>>>> HTML/JS/CSS from /_plugins/<pluginname>, I’m happy to >>>>>>>> add this to the branch. >>>>>>>> >>>>>>>> Best Jan -- >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -Russell >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Aug 2, 2013 at 2:17 PM, Jan Lehnardt >>>>>>>>> <j...@apache.org (mailto:j...@apache.org)> wrote: >>>>>>>>> >>>>>>>>>> And a few more (from COUCHDB-1867): >>>>>>>>>> >>>>>>>>>> - Add uninstall, incl. Futon UI. - Only install a >>>>>>>>>> plugin if the source and target CouchDB version >>>>>>>>>> matches. - Rebase against master. >>>>>>>>>> >>>>>>>>>> * * * >>>>>>>>>> >>>>>>>>>> This concludes my list for a Minimally Viable >>>>>>>>>> Plugin feature. (See the original email or >>>>>>>>>> README.md (http://README.md)* for the roadmap) >>>>>>>>>> >>>>>>>>>> I’d appreciate some more reviews & feedback**, but >>>>>>>>>> other than that, I’d be happy to ship this as an >>>>>>>>>> experimental feature in any next release. >>>>>>>>>> >>>>>>>>>> * >>>>>>>>>> https://github.com/janl/couchdb/blob/1867-feature-plugins/src/couch_plugins/README.md#roadmap >>> > ** >>>>>>>>>> https://github.com/janl/couchdb/compare/apache:master...1867-feature-plugins >>> > Best >>>>>>>>>> Jan -- >>>>>>>>>> >>>>>>>>>> On Aug 1, 2013, at 19:34 , Jan Lehnardt >>>>>>>>>> <j...@apache.org (mailto:j...@apache.org)> wrote: >>>>>>>>>> >>>>>>>>>>> A few updates: >>>>>>>>>>> >>>>>>>>>>> By Bob Ippolito / @etrepum: - Plugins are now >>>>>>>>>>> installed in libdir (instead of /tmp). - Config >>>>>>>>>>> loading is now done with proper .ini files. - >>>>>>>>>>> Various cleanups and code review (Thanks!). >>>>>>>>>>> >>>>>>>>>>> Mine (most suggested by Bob): - `plugins.html` >>>>>>>>>>> now shows you if a plugin is already installed. >>>>>>>>>>> and which version, if it doesn’t match the >>>>>>>>>>> installable one. - The Install button now >>>>>>>>>>> disables after an installation. - Plugins are now >>>>>>>>>>> registered with couch_config as >>>>>>>>>>> /_config/plugins/name = version - Updated >>>>>>>>>>> `couch-config` to print --erlang-version and >>>>>>>>>>> --erl-bin - Updated the geocouch plugin to use >>>>>>>>>>> the new options in `couch-config`. - Added Bob >>>>>>>>>>> Ippolito’s couchperuser plugin to Futon. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Best Jan -- >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Jul 31, 2013, at 19:07 , Jan Lehnardt >>>>>>>>>>> <j...@apache.org (mailto:j...@apache.org)> wrote: >>>>>>>>>>> >>>>>>>>>>>> Heya, >>>>>>>>>>>> >>>>>>>>>>>> I couldn’t help myself thinking about plugin >>>>>>>>>>>> stuff and ended up whipping up a proof of >>>>>>>>>>>> concept. >>>>>>>>>>>> >>>>>>>>>>>> Here’s a <1 minute demo video: >>>>>>>>>>>> >>>>>>>>>>>> https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.mov >>> > Alternative encoding: >>>>>>>>>>>> >>>>>>>>>>>> https://dl.dropboxusercontent.com/u/82149/couchdb-plugins-demo.m4v) >>> > In my head the whole plugin idea is a very wide area, but I was so >>>>>>>>>>>> intrigued by the idea of getting something >>>>>>>>>>>> running with a click on a button in Futon. So I >>>>>>>>>>>> looked for a minimally viable plugin system. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ## Design principles >>>>>>>>>>>> >>>>>>>>>>>> It took me a day to put this all together and >>>>>>>>>>>> this was only possible because I took a lot of >>>>>>>>>>>> shortcuts. I believe they are all viable for a >>>>>>>>>>>> first iteration of a plugins system: >>>>>>>>>>>> >>>>>>>>>>>> 1. Install with one click on a button in Futon >>>>>>>>>>>> (or HTTP call) 2. Only pure Erlang plugins are >>>>>>>>>>>> allowed. 3. The plugin author must provide a >>>>>>>>>>>> binary package for each Erlang (and, later, >>>>>>>>>>>> each CouchDB version). 4. Complete trust-based >>>>>>>>>>>> system. You trust me to not do any nasty things >>>>>>>>>>>> when you click on the install button. No >>>>>>>>>>>> crypto, no nothing. Only people who can commit >>>>>>>>>>>> to Futon can release new versions of plugins. >>>>>>>>>>>> 5. Minimal user-friendlyness: won’t install >>>>>>>>>>>> plugins that don’t match the current Erlang >>>>>>>>>>>> version, gives semi-sensible error messages >>>>>>>>>>>> (wrapped in a HTTP 500 response :) 6. Require >>>>>>>>>>>> a pretty strict format for binary releases. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ## Roadmap >>>>>>>>>>>> >>>>>>>>>>>> Here’s a list of things this first iterations >>>>>>>>>>>> does and doesn’t do: >>>>>>>>>>>> >>>>>>>>>>>> - Pure Erlang plugins only. No C-dependencies, >>>>>>>>>>>> no JavaScript, no >>>>>>>>>> nothing. >>>>>>>>>>>> - No C-dependencies. - Install a plugin via >>>>>>>>>>>> Futon (or HTTP call). Admin only. - A hardcoded >>>>>>>>>>>> list of plugins in Futon. - Loads a >>>>>>>>>>>> pre-packaged, pre-compiled .tar.gz file from a >>>>>>>>>>>> URL. - Only installs if Erlang version matches. >>>>>>>>>>>> - No security checking of binaries. - No >>>>>>>>>>>> identity checking of binaries. >>>>>>>>>>>> >>>>>>>>>>>> Here are a few things I want to add before I >>>>>>>>>>>> call it MVP*: >>>>>>>>>>>> >>>>>>>>>>>> - Uninstall a plugin via Futon (or HTTP call). >>>>>>>>>>>> Admin only. - Only installs if CouchDB version >>>>>>>>>>>> matches. - Binaries must be published on >>>>>>>>>>>> *.apache.org (http://apache.org). - Register >>>>>>>>>>>> installed plugins in the config system. - Make >>>>>>>>>>>> sure plugins start with the next restart of >>>>>>>>>>>> CouchDB. - Show when a particular plugin is >>>>>>>>>>>> installed. >>>>>>>>>>>> >>>>>>>>>>>> *MVP hopefully means you agree we can ship >>>>>>>>>>>> this with a few warnings so people can get a >>>>>>>>>>>> hang of it. >>>>>>>>>>>> >>>>>>>>>>>> Here is a rough list of features squared >>>>>>>>>>>> against future milestones: >>>>>>>>>>>> >>>>>>>>>>>> Milestone 2: Be creator friendly - Make it easy >>>>>>>>>>>> to build a CouchDB plugin by providing one or >>>>>>>>>>>> more easy to start templates. - Make it easy to >>>>>>>>>>>> publish new plugins and new versions of >>>>>>>>>>>> existing >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> plugins. >>>>>>>>>>>> - Make it easy to supply packages for multiple >>>>>>>>>>>> Erlang & CouchDB >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> versions. >>>>>>>>>>>> >>>>>>>>>>>> Milestone 3: Public registry - Instead of >>>>>>>>>>>> hardcoding a list of plugins into >>>>>>>>>>>> Futon/Fauxton, we load a list of applicable >>>>>>>>>>>> plugins from a central (and configurable) >>>>>>>>>>>> plugins repository. - This allows plugin >>>>>>>>>>>> authors to publish new plugins and new versions >>>>>>>>>>>> of existing plugins independently. >>>>>>>>>>>> >>>>>>>>>>>> Milestone 4: Other Languages - Figure out how >>>>>>>>>>>> to handle C-dependencies for Erlang plugins. - >>>>>>>>>>>> Figure out how to allow other language plugins >>>>>>>>>>>> (c.f. non-JS query servers) >>>>>>>>>>>> >>>>>>>>>>>> Milestone X: Later - Add some >>>>>>>>>>>> account/identity/maybe crypto-web-of-trust >>>>>>>>>>>> system for authors to publish “legit” plugins. >>>>>>>>>>>> - Sign & verify individual releases. >>>>>>>>>>>> >>>>>>>>>>>> A few more things that can happen concurrently >>>>>>>>>>>> depending on what plugins require: - Integrate >>>>>>>>>>>> Erlang/JS tests in the installation - >>>>>>>>>>>> Integrate docs >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ## How it works >>>>>>>>>>>> >>>>>>>>>>>> This plugin system lives in `src/couch_plugins` >>>>>>>>>>>> and is a tiny CouchDB module. >>>>>>>>>>>> >>>>>>>>>>>> It exposes one new API endpoint `/_plugins` >>>>>>>>>>>> that an admin user can POST to. >>>>>>>>>>>> >>>>>>>>>>>> The additional Futon page lives at >>>>>>>>>>>> /_utils/plugins.html it is hardcoded. >>>>>>>>>>>> >>>>>>>>>>>> Futon (or you) post an object to `/_plugins` >>>>>>>>>>>> with four properties: >>>>>>>>>>>> >>>>>>>>>>>> { "name": "geocouch", // name of the plugin, >>>>>>>>>>>> must be unique "url": >>>>>>>>>>>> "http://people.apache.org/~jan";, // “base URL” >>>>>>>>>>>> for plugin >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> releases (see below) >>>>>>>>>>>> "version": "couchdb1.2.x_v0.3.0-11-gd83ba22", >>>>>>>>>>>> // whatever version >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> internal to the plugin >>>>>>>>>>>> "checksums": { "R15B03": >>>>>>>>>>>> "ZetgdHj2bY2w37buulWVf3USOZs=" // base64’d sha >>>>>>>>>>>> hash >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> over the binary >>>>>>>>>>>> } } >>>>>>>>>>>> >>>>>>>>>>>> `couch_plugins` then attempts to download a >>>>>>>>>>>> .tar.gz from this location: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> http://people.apache.org/~jan/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03.tar.gz >>> > It should be obvious how the URL is constructed from the POST data. >>>>>>>>>>>> (This url is live, feel free to play around >>>>>>>>>>>> with this tarball). >>>>>>>>>>>> >>>>>>>>>>>> Next it calculates the sha hash for the >>>>>>>>>>>> downloaded .tar.gz file and matches it against >>>>>>>>>>>> the correct version in the `checksums` >>>>>>>>>>>> parameter. >>>>>>>>>>>> >>>>>>>>>>>> If that succeeds, we unpack the .tar.gz file >>>>>>>>>>>> (currently in `/tmp`, need to find a better >>>>>>>>>>>> place for this) and adds the extracted >>>>>>>>>>>> directory to the Erlang code path >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> (`code:add_path("/tmp/couchdb_plugins/geocouch-couchdb1.2.x_v0.3.0-12-g4ea0bea-R15B03/ebin")`) >>> > and loads the included application (`application:load(geocouch)`). >>>>>>>>>>>> >>>>>>>>>>>> Then it looks into the `./config` directory >>>>>>>>>>>> that lives next to `ebin/` in the plugin >>>>>>>>>>>> directory for a file `config.erlt` >>>>>>>>>>>> (“erl-terms”). with a list of configuration >>>>>>>>>>>> parameters to load. We parse the file and set >>>>>>>>>>>> the config directives one by one. >>>>>>>>>>>> >>>>>>>>>>>> If that all goes to plan, we report success >>>>>>>>>>>> back to the HTTP caller. >>>>>>>>>>>> >>>>>>>>>>>> That’s it! :) >>>>>>>>>>>> >>>>>>>>>>>> It’s deceptively simple, probably does a few >>>>>>>>>>>> things very wrong and leaves a few things open >>>>>>>>>>>> (see above). >>>>>>>>>>>> >>>>>>>>>>>> One open question I’d like an answer for is >>>>>>>>>>>> finding a good location to unpack & install the >>>>>>>>>>>> plugin files that isn’t `tmp`. If the answer is >>>>>>>>>>>> different for a pre-BigCouch/rcouch-merge and >>>>>>>>>>>> post-BigCouch/rcouch- merge world, I’d love to >>>>>>>>>>>> know :) >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> ## Code >>>>>>>>>>>> >>>>>>>>>>>> The main branch for this is >>>>>>>>>>>> 1867-feature-plugins: >>>>>>>>>>>> >>>>>>>>>>>> ASF: >>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=couchdb.git;a=log;h=refs/heads/1867-feature-plugins >>> > GitHub: https://github.com/janl/couchdb/compare/1867-feature-plugins >>>>>>>>>>>> >>>>>>>>>>>> I created a branch on GeoCouch that adds a few >>>>>>>>>>>> lines to its `Makefile` that shows how a >>>>>>>>>>>> binary package is built: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> https://github.com/janl/geocouch/compare/couchbase:couchdb1.3.x...couchdb1.3.x-plugins >>> > * * * >>>>>>>>>>>> >>>>>>>>>>>> I hope you like this :) Please comment and >>>>>>>>>>>> improve heavily! >>>>>>>>>>>> >>>>>>>>>>>> Let me know if you have any questions :) >>>>>>>>>>>> >>>>>>>>>>>> If you have any criticism, please phrase it in >>>>>>>>>>>> a way that we can use to improve this, thanks! >>>>>>>>>>>> >>>>>>>>>>>> Best, Jan -- > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.12 (GNU/Linux) > Comment: Using GnuPG with undefined - http://www.enigmail.net/ > > iQEcBAEBAgAGBQJR/ntiAAoJEAq942X94y8mpkYIAK1w9MBOmXSw3BZc9OJf55W1 > 4KRHvtvpyUy5BTZyoCHCeKquHDIy+rRJTnX69y3k4yXwEAG6kocJ5UldWUSn/tX8 > RAnEyIeTQs5YAjybutPIjpw6ClXgJnxON4sDCjvDPMoj3csE2RbxHqMD1udSGDd5 > OuUdqYBvC2g8XpNq3hbxtX7O0982s4OOfdJAywTpQ4vrcBhql+5m70YUiM+zpRF2 > JjQxngk9110aMEBDm6CgjW6z7XtwMbWqsXuFDfr5/RvFwBdAD+PksfWvPyUOmq2x > Bga7Dqa+clBN6AmA6oNGibXcIAjL//vqynFK4XKOSBchKuLurPypE5b30pkzmX4= > =S/C3 > -----END PGP SIGNATURE-----
