nickimho commented on issue #2329: URL: https://github.com/apache/couchdb/issues/2329#issuecomment-5484284807
@nickva Thanks for the clarification. That's a good (and more) complete point. To share more of our thoughts: This fix mainly solved an optimization we were not able to make previously: 1. We have been looking to optimize performance (how fast a query returns) and load (mostly backend work on the fabric). 2. The performance aspect is more intuitive, and before Egor's patch, we didn't have a good solution to limit the traffic on local zone for that metadata calls. 3. For the load aspect (assuming read heavy), we are evaluating: 3.a. For doc query: /db/_all_docs?include_docs=true&stable=true&limit=1&key=%22DOC_ID%22'. Even though the performance is similar to r=1, this is better than r=1 (for normal case) as it doesn't generate fabric read against all shards. 3.b. For View query: /db/_design/ddoc/_view/...?stable=true. With This avoids inter-zone with a big enough "[ddoc_cache] refresh_timeout" value against ddoc query once cached. This seems to be very beneficial for heavy read db's who ddocs are not changing once deployed. 3.c. The secondary benefit we are hoping to gain by limiting fabric traffic is (hopefully) CouchDB cluster (multi-zone setup) would to be able to handle "nightmare" network scenario (i.e. network flapping) better. 4. To balance out this approach (where shards replica have less chance of detecting out of sync and View index update might not get triggered in foreign zones), 4.a. We are considering to potentially mitigate this with an external process (e.g. a simple script that crawls all /db/_sync_shards over time) 4.b. Similarly, for larger databases (write heavy) where not all zones get View query equally, have an external process to "tickle" those Views in all zones. A lot more testing is still required on our end to validate this approach. Still, based on the current testing we have done with the patch and what we have observed through tcpdump, this approach seems reasonable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
