Hello all, I've started to put together some tools to test couchdb in a number of ways. Ideally this tool will provide a means to new users to better understand how couch will perform in their environments, additionally it could be used to assist then in tuning their system for expected loads.
When I first threw this idea out there I got a number of suggestion. One big time saver was to use a proven erlang tool for testing, so I have been looking into it. tsung (http://tsung.erlang-projects.org/ ), is a testing tool that can scale, and its written in erlang. It's really not to hard to get running, I'd recommend installing from the site and running though the build process. Some of the packed versions out there are very old and don't work. The documentation is weak. That said I did an exercise to get the xml required to run the tests. the results were much bigger than expected but futon does not just exercise the code but often requests object from the web server creating allot of background noise. That not really a problem as tsung does provide a nice little recorder for capturing the xml that needs to be written and now it just a matter of whipping it into the appropriate loops. The project can be found at: git://github.com/mcoolin/couchdb-stress.git Don't go running off their yet there's not much to see but some analysis documents and some rough ideas I've thrown together. Please email or add to the project if you can. I invite you to toss you ideas and suggestion for testing and testing profiles/approaches that you think would be helpful. I fully expect that there will be several testing files that treat the server in different ways, such as a read only vs logging vs read/write. multiple databases and multiple users on each. Tsung can be run on the same machine as couch, but it can also be placed on a server farm and produce millions of requests for stated periods of time. It is not my intention to repeat the futon test in tsung, I just used if to scrape of the xml I'll need for writing tests. But the exercise did reveal a few surprises. Some 7 sections failed, likely due to some special handling needed by tsung or maybe some feature in couchdb or timing issues. Testing was done against today's 1.2 version, please note that the trailing number refers to a page number of the results. I captured the logs of the errors and the calls that were made for those of you who are inclined to investigate. In the next few days I will try to get a sample going of at least one type of test. I'm pretty green with couchdb and tsung so if you have suggestions I'm all ears, I can create specific test suites that reveal warts with your help that hopefully in future will help those developing make couch even better. Testing recording of futon 1.2.0 with tsung-recorder 1 Basics - failed 6 XML 6 Logs 9 all_docs - passed 10 XML 10 Logs – NONE 16 attachments - passed 16 XML 16 Logs - None 31 31 attachments_multipart - passed 31 XML 31 Logs - None 45 45 attachment_names - Passed 45 XML 45 Logs - none 47 47 attachment_paths - passed 47 XML 47 Logs - none 51 attachment_ranges - passed 51 XML 51 Logs - none 52 attachment_views - passed 52 XML 52 Logs - none 57 57 auth_cache -passed 57 XML 57 Logs - none 63 63 batch_save - passed 63 XML 63 Logs - none 83 83 bulk_docs- passed 83 XML 83 Logs - none 86 86 Changes - passed 86 XML 86 Logs – none 94 Coffee - Passed 94 XML 94 Logs - none 96 Compact - failed – eventually passed timing? 96 XML 96 Logs 100 config - passed 120 XML 120 Logs - none 123 123 Conflicts - passed 123 XML 123 Logs - none 126 126 content_negotiation - passed 126 XML 126 Logs - none 127 127 cookie_auth - passed 127 XML 127 Logs - none 133 133 copy_doc - passed 133 XML 133 Logs - none 135 135 delayed_commits - failed 135 XML 135 Logs 140 391 design_docs - failed 391 XML 391 Logs 418 design_options - passed 457 XML 457 Logs - none 458 458 design_paths - passed 458 XML 458 Logs - none 460 460 erlang_views - passed 460 XML 460 Logs - none 556 556 etags_head - passed 556 XML 556 Logs- none 557 557 etags_views-failed 557 XML 557 Logs 564 658 form_submit - passed 658 XML 658 Logs - none 659 659 Http - passed 659 XML 659 Logs - none 660 660 invalid_docids - passed 660 XML 660 Logs - none 662 662 Jsonp - passed 662 XML 662 Logs - none 663 663 large_docs - passed 663 XML 663 Logs - none 676 676 list_views - passed 676 XML 676 Logs - none 681 681 lots_of_docs - passed 681 XML 681 Logs - none 686 686 method_override - passed 686 XML 686 Logs - none 687 687 multiple_rows - passed 687 XML 687 Logs - none 689 689 Oauth - passed 689 XML 689 Logs - none 698 Oauth_users_db - passed 698 XML 698 Logs - none 701 Proxyauth - passed 701 XML 701 Logs - none 704 704 Purge - passed 704 XML 704 Logs - none 708 708 reader_acl - failed 708 XML 708 Logs 713 797 recreate_doc - passed 797 XML 797 Logs - none 803 803 Reduce - passed 803 XML 803 Logs - none 809 809 reduce_builtin - passed 809 XML 809 Logs - none 866 866 reduce_false - passed 866 XML 866 Logs - none 868 868 reduce_false_temp - passed 868 XML 868 Logs - none 870 870 Replication - passed 870 XML 870 Logs - none 871 replicator_db - failed 871 XML 871 Logs 878 replicator_db_security - passed 970 XML 970 Logs - none 978 rev_stemming - passed 978 XML 978 Logs - none 982 982 Rewrite - passed 982 XML 982 Logs - none 988 988 security_validation - passed 988 XML 988 Logs - none 1000 1000 show_documents - passed 1000 XML 1000 Logs - none 1008 1008 Stats - fails 1008 XML 1008 Logs 1009 1021 update_documents - passed 1021 XML 1021 Logs - none 1024 1024 users_db - passed 1024 XML 1024 Logs - none 1027 users_db_security - passed 1027 XML 1027 Logs - none 1032 1032 Utf8 - passed 1032 XML 1032 Logs - none 1033 1033 Uuids - passed 1033 XML 1033 Logs - none 1035 1035 view_collation - passed 1035 XML 1035 Logs - none 1041 1041 view_collation_raw - passed 1041 XML 1041 Logs - none 1046 1046 view_conflicts - passed 1046 XML 1046 Logs - none 1047 1047 view_compaction - passed 1047 XML 1047 Logs - none 1171 1171 view_errors - passed 1171 XML 1171 Logs - none 1174 1174 view_include_docs - passed 1174 XML 1174 Logs - none 1179 1179 view_multi_key_all_docs - passed 1179 XML 1179 Logs - none 1184 1184 view_multi_key_design - passed 1184 XML 1184 Logs - none 1188 1188 view_multi_key_temp - passed 1188 XML 1188 Logs - none 1194 1194 view_offsets - passed 1194 XML 1194 Logs- none 1209 1209 view_pagination - passed 1209 XML 1209 Logs - none 1217 1217 view_sandboxing - passed 1217 XML 1217 Logs - none 1220 1220 view_update_seq - passed 1220 XML 1220 Logs - none 1228 1228 view_xml - passed 1228 XML 1228 Logs - passed 1230
