> On 12 Oct 2015, at 13:30, Dale Harvey <d...@arandomurl.com> wrote: > > Started with > > python dev/run -n 1 --with-admin-party-please & > > and export COUCH_HOST='http://127.0.0.1:15984' > > > On 12 October 2015 at 13:26, Jan Lehnardt <j...@apache.org> wrote: > >> >>> On 12 Oct 2015, at 13:15, Dale Harvey <d...@arandomurl.com> wrote: >>> >>>> - instead of fixed test db names like `test_suite_db` that are used in >>> almost every test, >>>> we now generate a random database name for each test. The JS tests >>> expected db >>>> deletion to be synchronous, and that’s no longer the case in 2.0. >> Instead >>> of >>>> error-prone polling to wait for db deletion, we just use new db for >> every >>> test. >>>> An added benefit now is that (some) tests can run in parallel. >>> >>> Little confused by this, PouchDB uses consistent names in its test suite >>> (we had random >>> ones for a while and moved away), we very much depend on the database >> being >>> deleted >>> on the request returning and dont seem to have any problems with it >> against >>> master. >> >> Does the PouchDB test suite runs against the 5986 port, or with an n=1 >> cluster? >> This is with the default n=3 cluster against 5986. >> >> Best >> Jan >> -- >> >>> >>> On 12 October 2015 at 12:21, Jan Lehnardt <j...@apache.org> wrote: >>> >>>> >>>>> On 11 Oct 2015, at 18:53, Sebastian Rothbucher < >>>> sebastianrothbuc...@googlemail.com> wrote: >>>>> >>>>> Hi Jan, >>>>> >>>>> I transferred some of my findings to your branch - pls. find a PR for >> the >>>>> first few tests submitted 2 your branch. I put in TODOs where either >>>> there >>>>> is most probably a bug (filed COUCHDB-2852 and COUCHDB-2853) or we have >>>> to >>>>> do some more work (like making config available). >>>> >>>> Excellent, thanks! :) >>>> >>>> >>>>> Will from the Pouch team had the great idea of using -n 1 to mitigate >>>>> Heisenbugs due to distribution. I adopted it and it did work quite >> well. >>>> It >>>>> might make sense to put this into the dev/run call as a general rule, >>>> esp. >>>>> when checking for 201 (VS 202) status codes. Probably you know more on >>>>> that. >>>> >>>> We should carefully evaluate this :) >>>> >>>> >>>>> Anyways: I think we have to keep using _config on the local port as >> many >>>>> tests (like auth tests) depend on changed configurations. BTW: w/ -n 1 >>>> it's >>>>> only one call, but calling 3 _config(s) migt also work. >>>>> Hopefully (I just arrived at some test w/ that recently) we can spare >>>>> ourselves the _compact thing. I think we'd have to compact every single >>>>> shard via local port, right? for the first tests, we can go w/out as it >>>>> seems. Maybe we should stick to that for a while. >>>> >>>> My concern with this is that as far as I care, the local port is >> irrelevant >>>> to how CouchDB works. Of course it isn’t 100% irrelevant, but the >> cluster >>>> port is our publicised API for 2.0 and it’s the one that replaces the >> 1.x >>>> API for people upgrading. So having any tests that run on the local port >>>> of 2.0 don’t help us in any way (that’s not to say it still useful to >> have >>>> tests against them for the moment). >>>> >>>> Many of the _config-dependent tests substitute out the _users or >>>> _replicator >>>> databases, maybe we can find a workaround for those by just using the >> stock >>>> databases. >>>> >>>> For the other tests, maybe we split them out into their own “test >> suites” >>>> that start their own test cluster in the configuration they need and >> shut >>>> it down afterwards. >>>> >>>> Best >>>> Jan >>>> -- >>>> >>>> >>>>> >>>>> Looking fw 2 your thoughts >>>>> >>>>> Best >>>>> Sebastian >>>>> >>>>> P.S:: Maybe we can adopt some stuff from >>>>> >>>> >> https://github.com/apache/couchdb/compare/master...sebastianrothbucher:clustertest >>>>> >>>>> P.P.S.: Will keep working on it, just not in the next few days, >>>>> unfortunately ;-( >>>>> >>>>> On Sun, Oct 11, 2015 at 12:00 PM, Jan Lehnardt <j...@apache.org> wrote: >>>>> >>>>>> (I need your help deciding what to do with a bunch of tests, jump to >>>>>> “Discussion” below if you don’t care about the details before.) >>>>>> >>>>>> * * * >>>>>> >>>>>> Hi all, >>>>>> >>>>>> I spent some more time getting the 1.x JavaScript tests running on the >>>>>> cluster port of 2.0. WIP here: >>>> https://github.com/apache/couchdb/pull/354 >>>>>> — I’ve gotten a lot further, but there is still a lot to do and >> everyone >>>>>> here can help, I’ve posted instructions on how to do so further down. >> No >>>>>> Erlang knowledge needed, this is ideal for folks who want to start >> with >>>>>> contributing to CouchDB. >>>>>> >>>>>> >>>>>> Notable changes: >>>>>> >>>>>> - instead of fixed test db names like `test_suite_db` that are used in >>>>>> almost every test, we now generate a random database name for each >> test. >>>>>> The JS tests expected db deletion to be synchronous, and that’s no >>>> longer >>>>>> the case in 2.0. Instead of error-prone polling to wait for db >>>> deletion, we >>>>>> just use new db for every test. An added benefit now is that (some) >>>> tests >>>>>> can run in parallel. >>>>>> >>>>>> - tests now clean up after themselves (incomplete): tests previously >>>> only >>>>>> collectively used three or four databases, and deleted and re-created >>>> them >>>>>> before each test was run. Now with the change of creating a new >> database >>>>>> per test, we also need to clean up databases at the end of a test, >>>> because >>>>>> otherwise we a) leave a lot of databases around and b) CouchDB runs >> out >>>> of >>>>>> file descriptors during the test run. The cleanup isn’t 100% done yet, >>>> so >>>>>> we should check for stray databases after JS test runs until we’ve got >>>> it >>>>>> cleaned. One particular aspect here is when a test fails, it never >>>> reaches >>>>>> the cleanup code, so we might have to wrap everything into a try/catch >>>>>> block. >>>>>> >>>>>> - I disabled all tests that still fail and made them so they print the >>>>>> reason why they are failing, so that fixing them should be a lot >> easier >>>>>> now. Some of them print just “TODO”, their error cause is yet to be >>>>>> determined. >>>>>> >>>>>> - couchdb.query() which previously executed temporary views, now under >>>> the >>>>>> hood creates a design doc with a random name and queries that view. >> This >>>>>> fixes all tests relying on temp views without having to change them >> all >>>>>> individually. This is because 2.0 doesn’t have temp views anymore >>>> (which is >>>>>> a good thing). >>>>>> >>>>>> - restartServer() is no longer used (mostly). >>>>>> >>>>>> I already filed a few bugs that showed things missing in the cluster >> API >>>>>> that we’d expect to be there: >>>>>> >>>>>> - https://issues.apache.org/jira/browse/COUCHDB-2849 >>>>>> - https://issues.apache.org/jira/browse/COUCHDB-2850 >>>>>> - https://issues.apache.org/jira/browse/COUCHDB-2851 >>>>>> >>>>>> Based on this I am ever more convinced we should get this test suite >>>> work >>>>>> against the 2.0 clustered API, because we’ll be finding some more >>>>>> inconsistencies that we should fix before we invite beta testers. >>>>>> >>>>>> >>>>>> ## Discussion: >>>>>> >>>>>> We have a whole number of tests that use the test-internal function >>>>>> run_on_modified_server. They use the _config API to test CouchDB in >>>>>> different configurations from the default one. All of these tests now >>>> throw >>>>>> an error that _config isn’t available on the clustered API. How should >>>> we >>>>>> handle them? >>>>>> >>>>>> The tests are: >>>>>> >>>>>> - erlang_views.js >>>>>> - jsonp.js >>>>>> - oauth_users_db.js >>>>>> - oauth_users_db.js >>>>>> - proxyauth.js >>>>>> - reader_acl.js >>>>>> - rewrite.js >>>>>> - security_validation.js >>>>>> - show_documents.js >>>>>> - users_db.js >>>>>> - view_errors.js >>>>>> >>>>>> >>>>>> And a few tests aside from the proper compaction tests call _compact >>>> which >>>>>> is also not available on the cluster. What to do with them? >>>>>> >>>>>> - recreate_doc.js >>>>>> - and a few more that are currently disabled for other reasons, sorry >> I >>>>>> don’t have a full list at this point. >>>>>> >>>>>> * * * >>>>>> >>>>>> >>>>>> Bottom line: It’d be great if other folks here could join in the fun. >> No >>>>>> Erlang needed, this is purely JavaScript and CouchDB HTTP API-work :) >>>>>> >>>>>> >>>>>> ## How to help: >>>>>> >>>>>> 1. Check out https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip >>>>>> 2. run `make clean && ./configure -c --disable-docs --disable-fauxton >> && >>>>>> make` >>>>>> 3. `make javascript` runs the whole test suite >>>>>> >>>>>> To run the whole test suite without waiting for rebar to confirm that >> no >>>>>> Erlang code needs to be compiled, run `./test/javascript/run` for the >>>> whole >>>>>> test suite, or `./test/javascript/run all_docs` to run an individual >>>> test >>>>>> (all_docs runs test/javascript/tests/all_docs.js, other file names >> match >>>>>> accordingly). >>>>>> >>>>>> Feel free to send me PRs against the branch on >>>>>> https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip, I’ll merge >>>>>> them, so the PR on apache/couchdb gets updated. >>>>>> >>>>>> I’m looking forward to your contributions! :) >>>>>> >>>>>> If you have any questions or have trouble getting started or just >> would >>>>>> like some hand-holding, let me know! >>>>>> >>>>>> >>>>>> Best >>>>>> Jan >>>>>> -- >>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> -- >>>> Professional Support for Apache CouchDB: >>>> http://www.neighbourhood.ie/couchdb-support/ >>>> >>>> >> >> -- >> Professional Support for Apache CouchDB: >> http://www.neighbourhood.ie/couchdb-support/ >> >>
-- Professional Support for Apache CouchDB: http://www.neighbourhood.ie/couchdb-support/
