[jira] [Commented] (COUCHDB-1953) Speed up parsing of multipart/related requests
[ https://issues.apache.org/jira/browse/COUCHDB-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881784#comment-13881784 ] Nick North commented on COUCHDB-1953: - Many thanks everyone. I'll polish off anything else that needs to be done and push to master later today (if no-one has beaten me to it). Is it worth including this in 1.6, as it turns out to fix a reported problem? I don't want to disrupt the release process so am happy not to, if it's not appropriate. > Speed up parsing of multipart/related requests > -- > > Key: COUCHDB-1953 > URL: https://issues.apache.org/jira/browse/COUCHDB-1953 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Reporter: Nick North > Attachments: tests1953.erl > > > Parsing of multipart/related requests searches for the MIME boundary string > using the couch_httpd:find_in_binary/2 function, which can be made more > efficient. > When the boundary string is not found in its entirety in the search data, the > function should then look to see if the data ends with a prefix of the > string, but it currently looks for any prefix of the string almost anywhere > in the search data. > A pull request to fix this will be submitted shortly. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-1953) Speed up parsing of multipart/related requests
[ https://issues.apache.org/jira/browse/COUCHDB-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881800#comment-13881800 ] Benoit Chesneau commented on COUCHDB-1953: -- I'm +1 to have it included in 1.6. If the relation with #COUCHDB-1986 is confirmed It's a must have anyway. > Speed up parsing of multipart/related requests > -- > > Key: COUCHDB-1953 > URL: https://issues.apache.org/jira/browse/COUCHDB-1953 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Reporter: Nick North > Attachments: tests1953.erl > > > Parsing of multipart/related requests searches for the MIME boundary string > using the couch_httpd:find_in_binary/2 function, which can be made more > efficient. > When the boundary string is not found in its entirety in the search data, the > function should then look to see if the data ends with a prefix of the > string, but it currently looks for any prefix of the string almost anywhere > in the search data. > A pull request to fix this will be submitted shortly. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-1953) Speed up parsing of multipart/related requests
[ https://issues.apache.org/jira/browse/COUCHDB-1953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881946#comment-13881946 ] ASF subversion and git services commented on COUCHDB-1953: -- Commit 0bf823c6e5d6a7054e7cb1979eabc7695c2707e2 in branch refs/heads/master from [~nicknorth] [ https://git-wip-us.apache.org/repos/asf?p=couchdb.git;h=0bf823c ] Speed up and move couch_httpd:find_in_binary. See https://issues.apache.org/jira/browse/COUCHDB-1953 > Speed up parsing of multipart/related requests > -- > > Key: COUCHDB-1953 > URL: https://issues.apache.org/jira/browse/COUCHDB-1953 > Project: CouchDB > Issue Type: Improvement > Components: HTTP Interface >Reporter: Nick North > Attachments: tests1953.erl > > > Parsing of multipart/related requests searches for the MIME boundary string > using the couch_httpd:find_in_binary/2 function, which can be made more > efficient. > When the boundary string is not found in its entirety in the search data, the > function should then look to see if the data ends with a prefix of the > string, but it currently looks for any prefix of the string almost anywhere > in the search data. > A pull request to fix this will be submitted shortly. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
couchdb pull request: Speed up couch_httpd:find_in_binary.
Github user NorthNick closed the pull request at: https://github.com/apache/couchdb/pull/115
[SOLUTION :) ] Top posting in threads on the list's
Hi everyone, two days ago I wrote an email to all lists regarding (me ranting a bit - sorry) top posting in threads. The resulting discussion was a summary of many different opinions about how to post. I try to summarise this a bit: / yes, top posting is evil - we should not do it / top posting is ok, I do that since many years / combine top posting, inline posting and bottom posting / I don't care, my email client will fix it for me / we should not give any advise or guidelines Well, my conclusion is to not give any advise but my personal idea about it. I like to write inline comments and more comments or topics at the end of an post. I think it's the best way to read it. I use exclusively GMail on a laptop (no smartphone). BUT - this is my personal opinion and as far as I understand, it was mostly ok, how the threads are structured now. So everybody is welcome to keep doing how he does. What I really don't want is to keep people out of discussions because they are afraid of doing comments or posting wrong. Please don't be afraid! I was really worried when this came up. Sorry for that. I hope anybody is now feeling comfortable again. Let's keep on going with this really harmonious lists. Thanks and have a nice weekend Andy -- Andy Wenk Hamburg - Germany RockIt! http://www.couchdb-buch.de http://www.pg-praxisbuch.de GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588 https://people.apache.org/keys/committer/andywenk.asc
[jira] [Commented] (COUCHDB-2034) authentication_handlers does not accept complex arguments
[
https://issues.apache.org/jira/browse/COUCHDB-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882109#comment-13882109
]
Daniel Moore commented on COUCHDB-2034:
---
Ah, I've been wondering how CouchDB picked names for my handlers...
How would you specify the order of handlers that way? I thought the config
system was more of a bag than a list; would you specify a list elsewhere made
up of strings of the auth handlers to use?
> authentication_handlers does not accept complex arguments
> -
>
> Key: COUCHDB-2034
> URL: https://issues.apache.org/jira/browse/COUCHDB-2034
> Project: CouchDB
> Issue Type: Bug
>Reporter: Daniel Moore
>Assignee: Adam Kocoloski
>
> I have written a custom authentication handler for couch, but when I use the
> "SpecArg" syntax with embedded tuples, I get parse errors spinning up the
> server. For instance, a config like:
> {code}
> [httpd]
> authentication_handlers = {my_mod, my_fun, [ { one, two }, { three, four } ] }
> {code}
> Gives:
> {code}
> [error] [<0.93.0>] {error_report,<0.30.0>,
> {<0.93.0>,supervisor_report,
> [{supervisor,{local,couch_secondary_services}},
> {errorContext,start_error},
> {reason,
>{'EXIT',
> {{case_clause,
> {error,
>{1,erl_parse,["syntax error before: ","'.'"]}}},
> [{couch_httpd,make_arity_1_fun,1,
>[{file,"couch_httpd.erl"},{line,200}]},
> {couch_httpd,'-set_auth_handlers/0-fun-0-',1,
>[{file,"couch_httpd.erl"},{line,194}]},
> {lists,map,2,[{file,"lists.erl"},{line,1224}]},
> {couch_httpd,set_auth_handlers,0,
>[{file,"couch_httpd.erl"},{line,193}]},
> {couch_httpd,start_link,2,
>[{file,"couch_httpd.erl"},{line,130}]},
> {supervisor,do_start_child,2,
>[{file,"supervisor.erl"},{line,310}]},
> {supervisor,start_children,3,
>[{file,"supervisor.erl"},{line,293}]},
> {supervisor,init_children,2,
>[{file,"supervisor.erl"},{line,259}]}]}}},
> {offender,
>[{pid,undefined},
> {name,httpd},
> {mfargs,{couch_httpd,start_link,[]}},
> {restart_type,permanent},
> {shutdown,brutal_kill},
> {child_type,worker}]}]}}
> {code}
> This seems to be as a result of [using a
> regex|https://github.com/apache/couchdb/blob/master/src/couchdb/couch_httpd.erl?source=c#L224-L225]
> to split the tuples. Perhaps we could change the strategy to wrapping the
> string with "\[" and "\]" and parsing it altogether?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (COUCHDB-1986) 04-replication-large-atts.t times out
[ https://issues.apache.org/jira/browse/COUCHDB-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882113#comment-13882113 ] Andy Wenk commented on COUCHDB-1986: I ran make check against master (6fd38404137e2c61) and have stil the problem: /Users/andwen/project/couchdb/couchdb-src-clean/src/couch_replicator/test/04-replication-large-atts.t ... 493/? Bailout called. Further testing stopped: Timeout waiting for replication to finish FAILED--Further testing stopped: Timeout waiting for replication to finish make[2]: *** [check] Error 127 make[1]: *** [check-recursive] Error 1 make: *** [check-recursive] Error 1 > 04-replication-large-atts.t times out > - > > Key: COUCHDB-1986 > URL: https://issues.apache.org/jira/browse/COUCHDB-1986 > Project: CouchDB > Issue Type: Bug > Components: Replication >Affects Versions: 1.5.0 >Reporter: Jan Lehnardt > > 04-replication-large-atts.t gets stuck around 558, sometimes a little earlier > or later, but it times out eventually, regardless of the timeout. I tried > doubling and such. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2034) authentication_handlers does not accept complex arguments
[
https://issues.apache.org/jira/browse/COUCHDB-2034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882127#comment-13882127
]
Adam Kocoloski commented on COUCHDB-2034:
-
You're quite right [~danielmoore]; I came to the same realization the morning
after I left this edit but never noted it on the ticket. I still think it would
be preferable to have explicit labels and separate out the definitions, so e.g.
{code}
[httpd]
authentication_handlers = my_auth, basic, cookie, oauth
[httpd.authentication_handlers]
basic = {couch_httpd_auth, default_authentication_handler}
cookie = {couch_httpd_auth, cookie_authentication_handler}
oauth = {couch_httpd_oauth, oauth_authentication_handler}
my_auth = {my_mod, my_fun, [ { one, two }, { three, four } ] }
{code}
which has the advantage of being backwards compatible with current syntax if we
want it to be.
> authentication_handlers does not accept complex arguments
> -
>
> Key: COUCHDB-2034
> URL: https://issues.apache.org/jira/browse/COUCHDB-2034
> Project: CouchDB
> Issue Type: Bug
>Reporter: Daniel Moore
>Assignee: Adam Kocoloski
>
> I have written a custom authentication handler for couch, but when I use the
> "SpecArg" syntax with embedded tuples, I get parse errors spinning up the
> server. For instance, a config like:
> {code}
> [httpd]
> authentication_handlers = {my_mod, my_fun, [ { one, two }, { three, four } ] }
> {code}
> Gives:
> {code}
> [error] [<0.93.0>] {error_report,<0.30.0>,
> {<0.93.0>,supervisor_report,
> [{supervisor,{local,couch_secondary_services}},
> {errorContext,start_error},
> {reason,
>{'EXIT',
> {{case_clause,
> {error,
>{1,erl_parse,["syntax error before: ","'.'"]}}},
> [{couch_httpd,make_arity_1_fun,1,
>[{file,"couch_httpd.erl"},{line,200}]},
> {couch_httpd,'-set_auth_handlers/0-fun-0-',1,
>[{file,"couch_httpd.erl"},{line,194}]},
> {lists,map,2,[{file,"lists.erl"},{line,1224}]},
> {couch_httpd,set_auth_handlers,0,
>[{file,"couch_httpd.erl"},{line,193}]},
> {couch_httpd,start_link,2,
>[{file,"couch_httpd.erl"},{line,130}]},
> {supervisor,do_start_child,2,
>[{file,"supervisor.erl"},{line,310}]},
> {supervisor,start_children,3,
>[{file,"supervisor.erl"},{line,293}]},
> {supervisor,init_children,2,
>[{file,"supervisor.erl"},{line,259}]}]}}},
> {offender,
>[{pid,undefined},
> {name,httpd},
> {mfargs,{couch_httpd,start_link,[]}},
> {restart_type,permanent},
> {shutdown,brutal_kill},
> {child_type,worker}]}]}}
> {code}
> This seems to be as a result of [using a
> regex|https://github.com/apache/couchdb/blob/master/src/couchdb/couch_httpd.erl?source=c#L224-L225]
> to split the tuples. Perhaps we could change the strategy to wrapping the
> string with "\[" and "\]" and parsing it altogether?
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Created] (COUCHDB-2037) view unavailability on production design docs
Isaac Z. Schlueter created COUCHDB-2037: --- Summary: view unavailability on production design docs Key: COUCHDB-2037 URL: https://issues.apache.org/jira/browse/COUCHDB-2037 Project: CouchDB Issue Type: New Feature Components: JavaScript View Server Reporter: Isaac Z. Schlueter It can take a very long time to generate views for a large database. If you have an application that relies on those views, even if performance is acceptable, it is generally a serious problem if something causes the views for that design doc to no longer be available. This gets especially tricky if a new view is added or modified, as then all view data is thrown away and regenerated, causing your website or app to be unavailable for hours at a time. To work around this problem, I've taken to the following work flow: 1. PUT the new design doc to /db/_design/scratch 2. Load a view from /db/_design/scratch 3. Once view generation on _design/scratch finishes, send an http COPY request to move the _design/scratch to _design/app. This mostly works pretty well. However, if you have jobs that copy couchdb documents from one place to another, it can be very easy to accidentally trigger a view regeneration and cause your website or app to go down. (Qv today's npmjs.org outage.) I would love it if there was a way to say, "NEVER make the views of _design/app unavailable, even if they are temporarily out of date". I can imagine a few ways that this could be accomplished: 1. PUTs to the specified design doc fail if they modify the "views" member. 2. COPYs to the specified design doc fail if the COPY source does not have a full set of pre-generated views. 3. Views for the specified design doc are always done in the background, and stale=ok is assumed for all requests. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2037) view unavailability on production design docs
[ https://issues.apache.org/jira/browse/COUCHDB-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882159#comment-13882159 ] Jason Smith commented on COUCHDB-2037: -- Interestingly, CouchDB already has that view data available, even when you replace the design document. You must explicitly do a view cleanup to remove those files from disk: http://docs.couchdb.org/en/latest/api/database/compact.html?highlight=cleanup#post--db-_view_cleanup So this implementation is not as difficult as it may seem. A very quick and dirty fix could be a hidden or obscure URL where you could query a view directly by its hash value (the MD5 of the ddoc.views value, i.e. the name of the file on disk with the .view suffix), and couch could just trust that you know what you are talking about and attempt to query that file. Standard "thinking aloud" disclaimer applies :) > view unavailability on production design docs > - > > Key: COUCHDB-2037 > URL: https://issues.apache.org/jira/browse/COUCHDB-2037 > Project: CouchDB > Issue Type: New Feature > Components: JavaScript View Server >Reporter: Isaac Z. Schlueter > > It can take a very long time to generate views for a large database. If you > have an application that relies on those views, even if performance is > acceptable, it is generally a serious problem if something causes the views > for that design doc to no longer be available. This gets especially tricky > if a new view is added or modified, as then all view data is thrown away and > regenerated, causing your website or app to be unavailable for hours at a > time. > To work around this problem, I've taken to the following work flow: > 1. PUT the new design doc to /db/_design/scratch > 2. Load a view from /db/_design/scratch > 3. Once view generation on _design/scratch finishes, send an http COPY > request to move the _design/scratch to _design/app. > This mostly works pretty well. However, if you have jobs that copy couchdb > documents from one place to another, it can be very easy to accidentally > trigger a view regeneration and cause your website or app to go down. (Qv > today's npmjs.org outage.) > I would love it if there was a way to say, "NEVER make the views of > _design/app unavailable, even if they are temporarily out of date". I can > imagine a few ways that this could be accomplished: > 1. PUTs to the specified design doc fail if they modify the "views" member. > 2. COPYs to the specified design doc fail if the COPY source does not have a > full set of pre-generated views. > 3. Views for the specified design doc are always done in the background, and > stale=ok is assumed for all requests. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (COUCHDB-2037) view unavailability on production design docs
[ https://issues.apache.org/jira/browse/COUCHDB-2037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13882160#comment-13882160 ] Jason Smith commented on COUCHDB-2037: -- I wrote up some operational advice about this issue http://mail-archives.apache.org/mod_mbox/couchdb-user/201308.mbox/%3ccacly94uq9eafd_q-lxv8b3acipnhgmnawdn2c1ljnvumq-c...@mail.gmail.com%3E Surely you know this already but the unspoken general point is that human activity should *never* be able to modify production design documents. If that includes replication or replication-like activity, then you should be replicating _design/foo-staging docs and then manually upgrading them after a human confirms stuff. Bleh. This is just stupid platitudes. > view unavailability on production design docs > - > > Key: COUCHDB-2037 > URL: https://issues.apache.org/jira/browse/COUCHDB-2037 > Project: CouchDB > Issue Type: New Feature > Components: JavaScript View Server >Reporter: Isaac Z. Schlueter > > It can take a very long time to generate views for a large database. If you > have an application that relies on those views, even if performance is > acceptable, it is generally a serious problem if something causes the views > for that design doc to no longer be available. This gets especially tricky > if a new view is added or modified, as then all view data is thrown away and > regenerated, causing your website or app to be unavailable for hours at a > time. > To work around this problem, I've taken to the following work flow: > 1. PUT the new design doc to /db/_design/scratch > 2. Load a view from /db/_design/scratch > 3. Once view generation on _design/scratch finishes, send an http COPY > request to move the _design/scratch to _design/app. > This mostly works pretty well. However, if you have jobs that copy couchdb > documents from one place to another, it can be very easy to accidentally > trigger a view regeneration and cause your website or app to go down. (Qv > today's npmjs.org outage.) > I would love it if there was a way to say, "NEVER make the views of > _design/app unavailable, even if they are temporarily out of date". I can > imagine a few ways that this could be accomplished: > 1. PUTs to the specified design doc fail if they modify the "views" member. > 2. COPYs to the specified design doc fail if the COPY source does not have a > full set of pre-generated views. > 3. Views for the specified design doc are always done in the background, and > stale=ok is assumed for all requests. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
