On Mon, Jul 7, 2014 at 3:16 AM, Vinicius Assef <vinicius...@gmail.com> wrote:
> Ricardo, very nice your solution. > > Below, I wrote some questions about it. > > > > > On Sunday, 6 July 2014 02:01:23 UTC-5, Ricardo Pedroso wrote: > >> > >> - search is powered by whoosh and bottle - Web2py is querying through a > >> simple restful api. > > I was thinking about this piece of your architecture. > If your request goes through Web2py, why delegate search to bottle? > (and, also, wait for another request) > > Why not just using Whoosh inside Web2py? > Mainly for two reasons and because my vps has a small amount of memory available but has 8 cores. 1. I was worried about performance of whoosh. Whoosh is used in two different parts of the site, the search in the main page and the "maybe related" when viewing an article (eg: http://feeds.uni.me/feeds/default/article/281126?l=en&s=) and this are the two slowest (but still fast) things in the site. So this way I can put another core to work and I think I can release another greenlet when waiting for whoosh results and handle more concurrency since I'm using eventlet.monkey_patch() and from web2py I do this for the search: u = urllib2.urlopen('http://localhost:8080/search/' + str(int(p)) + '/' + arg + '?l=' + lang, timeout=15) json.loads(u.read()) 2. memory consumption of whoosh. Currently I have this memory usage: web2py - 25MB search (bottle+whoosh) - 39MB the background job that grabs and process the feeds - usually 40MB (when running) nginx master - 5MB nginx worker - 5.5MB - The site is run only with one process/thread since I'm using greenlet's through eventlet. Sometimes(not too often) the background job give me a crash with MemoryError so the first thing I put down to free some memory is the bottle+whoosh process and the rest of the site still works, of course, without the "search" and "maybe related" features. Another, less important, reason was that I like to complicate, it's funnier :) >> > >> - a small VPS with 256Mb of RAM. > > All these described processes run inside this vps? > yes. Ricardo -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list (Report Issues) --- You received this message because you are subscribed to the Google Groups "web2py-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to web2py+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
