Re: [Twisted-Python] Using a custom reactor in twisted trial for test cases?
On 2 Nov 2009, at 07:22, Crispin Wellington wrote: > On Mon, 2009-11-02 at 01:12 -0500, Glyph Lefkowitz wrote: >> On Nov 1, 2009, at 11:17 PM, Crispin Wellington wrote: > >>> I tried using LoopingCall, but it does not work. It only calls the >>> scheduler once. I think this has to do with the fact that the >>> stackless >>> scheduler needs to be interwoven with the twisted reactor pump. > >> What do you mean "does not work"? > > OK. Having a closer look, its not that looping call doesn't work, its > that there is some unknown number of "reactor pumps" between starting > the test, and finishing it. What I need is a way for the reactor to be > pumping away while a particular test function of a testcase continues > working. [snip] > Running this under trial, it just hangs, inside the while task.alive: > loop. > > So I guess my problem is my approach. How do I test long running > "tasklets" that use twisted calls (unlike this contrived example that > only sleeps) within the twisted trial framework? I'm no expert, but I have some tests against objects that use spawnProcess() which has the similar problem that I need to pump the reactor. The key is to return a deferred from the trial test method - then trial does the pump for you. I'm actually using @inlineCallbacks and the sleep() function mentioned in a previous post - you'll probably want task.deferLater() or similar. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Connect to an IRC server, modules.
Greetings,
first of all, I apologize for bad english. I am new with Twisted, and
although it seems very powerful, I'm having some problems in using it.
I'm trying to create an IRC bot, which basically does nothing. My idea
is to write a skeleton that's improved by modules, which do different
things in an IRC channel.
When some events happen some modules become active, so that they can
send messages, or notices, or private messages, and all what an IRC
bot can do; so, every module has to have complete access to the
methods of the bot class, but in the same time I need to keep the two
things separated, to improve modularity.
I used an example, called LogBot example. I have a .py file (I'll call
it "mainfile), in which the LogBot class is located, where the reactor
is set up etc, then I have a dir called modules, where modules are.
- mainfile.py
- modules
- modulea
- modulea.py
- moduleb
- moduleb.py
(I omitted __init__.py files).
Now the question is: how I can make accessible all the methods of the
LogBot class from EVERY .py inside the modules directory? I tried to
do the following:
In the mainfile, inside the LogBot class, I have
def joined(self, channel):
"""This will get called when the bot joins the channel."""
self.logger.log("[I have joined %s]" % channel)
This method is called when the bot joins the channel "channel".
Know, if I want to use modulea to greet user "x" when I enter the
channel, I would write:
def joined(self, channel):
"""This will get called when the bot joins the channel."""
self.logger.log("[I have joined %s]" % channel)
modulea.greet("x", "Hi, user x")
where greet is a function inside the module modulea (imported
somewhere in my mainfile).
Anyway, modulea is defined as following:
def greet(who, message):
## But here, how can the function know the method to use?
The solution can be...
##In mainfile.py
def joined(self, channel):
"""This will get called when the bot joins the channel."""
self.logger.log("[I have joined %s]" % channel)
modulea.greet(self, "x", "Hi, user x")
##In module a:
def greet(bot, who, message):
bot.msg(who, message)
But this does not work.
I tried to explain as better as I could. If you can, please use
examples and speak easy (can't understand very well).
Hope you can help me!
Thank you, in advance.
--Alberto Piu
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] reverse web proxy & threads
Hello!
I'm playing with reverse proxy from examples
(Twisted-8.2.0/doc/web/examples/proxy.py). May somebody explain what
part of code uses threads there?
pstree -p | grep python
-python(12786)-+-{python}(12795)
|-{python}(13050)
|-{python}(15547)
`-{python}(20767)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Ticket #3987 Review
On Oct 29, 2009, at 11:05 PM, exar...@twistedmatrix.com wrote: >> Well...I've now spent some actual time with 886 (-2,-3,and -4), and >> also with 3987. >> >> While there is some very nice stuff in there, I'm really not sure how >> one would actually put it to use since the back-end and the front-end >> seem to have been split into separate branches. > > They're not really split. The 3987 branch is complete and usable on > its > own. You don't need a checkout of the 886 branch. Ok, I have re-checked out just the 3987 branch without messing with 886. Using it as a direct substitution in a Django application using Twisted's WSGI, it completely breaks my application in ways that look suspiciously like the 8.2.0 release's WSGI problems that have since been fixed in Twisted trunk. Is 3987 based on 8.2.0 and, if so, what is the procedure for merging the trunk back to this branch (or visa versa, if that would be better). If 3987 is supposed to be in sync with trunk, I'll have to hunt down the WSGI problems I'm having because it's definitely not working in an application that works just fine with Twisted trunk. S ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Using a custom reactor in twisted trial for test cases?
Crispin: >OK. Having a closer look, its not that looping call doesn't work, its >that there is some unknown number of "reactor pumps" between starting >the test, and finishing it. What I need is a way for the reactor to be >pumping away while a particular test function of a testcase continues >working. As an example, here is a non-working test case. Notice the >comment "#what to do here to pump the reactor?". ... >while task.alive: >pass#what to do here to pump the reactor? This won't work because as long as the tasklet is alive (scheduled, paused, or blocked), you will be in a tight loop. And if that function is not running in the reactor, then the reactor will starve. If you are waiting to see if the tasklet finished, write a synchronizer. Cheers, Andrew ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Using a custom reactor in twisted trial for test cases?
Hello Glyph: >These examples aren't the greatest, because they tend to assume that >you _always_ have Stackless code that's ready to run, and you want to >run at some number of "frames per second". In 2007, folks (especially myself) are still trying to figure out if Stackless can play with Twisted correctly. I know my stuff in the repository is old. > For simple examples this makes sense, but for a system architecture >this is a limiting approach. If your stackless application wants to >sit idle and wait for input, you're still going to wake up once every >1/30 >second and checking to see if there's anything to do, burning CPU >cycles and >battery life. An approach would be for the reactor tasklet to look at the runnable list to see if there are any entries. If there are no entries (i.e., tasklets are blocked and dependent on the reactor tasklet), then the reactor tasklet can stop the loopingCall. This is easy enough to do. However I suspect one starts to get into custom reactor land. >Also, if you want to run *faster* than the arbitrary timeout you've >selected (1/30 second can be a very long time, especially if you're >doing something pseudo-realtime, like audio playback) you're out of luck. I believe there are two scenarios here: 1) no blocked tasklets and only runnable ones. If the runnable tasklets make no IO requests, it should be possible to put the reactor tasklet to sleep. Let the system run as fast as it can. 2) There is a mix of runnable (scheduled) and blocked tasklets. This scenario requires a more sophisticated scheduler that is trying to compute an optimal loopingCall value. This scenario is tricky. >Better would be to have tasklets schedule *themselves* for when they >want to run. In the Stackless world, tasklets start running from the moment they are created/inserted into the runnable list. Stackless in non-preemptive mode is more about knowing when to make a tasklet yield to the scheduler. Cheers, Andrew ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Ticket #3987 Review
On 03:20 pm, sstein...@gmail.com wrote: >On Oct 29, 2009, at 11:05 PM, exar...@twistedmatrix.com wrote: >>>Well...I've now spent some actual time with 886 (-2,-3,and -4), and >>>also with 3987. >>> >>>While there is some very nice stuff in there, I'm really not sure how >>>one would actually put it to use since the back-end and the front-end >>>seem to have been split into separate branches. >> >>They're not really split. The 3987 branch is complete and usable on >>its >>own. You don't need a checkout of the 886 branch. > >Ok, I have re-checked out just the 3987 branch without messing with >886. > >Using it as a direct substitution in a Django application using >Twisted's WSGI, it completely breaks my application in ways that look >suspiciously like the 8.2.0 release's WSGI problems that have since >been fixed in Twisted trunk. > >Is 3987 based on 8.2.0 and, if so, what is the procedure for merging >the trunk back to this branch (or visa versa, if that would be better). > >If 3987 is supposed to be in sync with trunk, I'll have to hunt down >the WSGI problems I'm having because it's definitely not working in an >application that works just fine with Twisted trunk. It's not based on 8.2.0, but neither is it in "sync" with trunk. It is a copy of trunk from r27271. You may indeed find that there are a number of bug fixes or feature additions which are present in tr...@head which are not present in the #3987 branch. If you want everything from tr...@head and the #3987 branch, then you may now be happy you did all that stuff with git. Merge the #3987 branch into tr...@head and you'll get everything from tr...@head and everything from the branch (that's tautological, sorry, I hope you know what I mean). If you're not sure what to do with git, then here's what to do with svn: BRANCHES=svn://svn.twistedmatrix.com/svn/Twisted/branches EXPRESSIVE=$BRANCHES/expressive-http-client-886-4 HIGHLEVEL=$BRANCHES/high-level-web-client-3987/ cd path/to/Twisted/trunk svn merge $express...@27272 express...@head svn merge $highle...@27275 $highle...@head Or using Combinator: chbranch Twisted expressive-http-client-886-4 unbranch Twisted chbranch Twisted high-level-web-client-3987 unbranch Twisted --force Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Ticket #3987 Review
On Nov 2, 2009, at 6:07 PM, exar...@twistedmatrix.com wrote: > On 03:20 pm, sstein...@gmail.com wrote: >> On Oct 29, 2009, at 11:05 PM, exar...@twistedmatrix.com wrote: Well...I've now spent some actual time with 886 (-2,-3,and -4), and also with 3987. While there is some very nice stuff in there, I'm really not sure how one would actually put it to use since the back-end and the front- end seem to have been split into separate branches. >>> >>> They're not really split. The 3987 branch is complete and usable on >>> its >>> own. You don't need a checkout of the 886 branch. >> >> Ok, I have re-checked out just the 3987 branch without messing with >> 886. >> >> Using it as a direct substitution in a Django application using >> Twisted's WSGI, it completely breaks my application in ways that look >> suspiciously like the 8.2.0 release's WSGI problems that have since >> been fixed in Twisted trunk. >> >> Is 3987 based on 8.2.0 and, if so, what is the procedure for merging >> the trunk back to this branch (or visa versa, if that would be >> better). >> >> If 3987 is supposed to be in sync with trunk, I'll have to hunt down >> the WSGI problems I'm having because it's definitely not working in >> an >> application that works just fine with Twisted trunk. > > It's not based on 8.2.0, but neither is it in "sync" with trunk. It > is > a copy of trunk from r27271. You may indeed find that there are a > number of bug fixes or feature additions which are present in > tr...@head > which are not present in the #3987 branch. > > If you want everything from tr...@head and the #3987 branch, then you > may now be happy you did all that stuff with git. Merge the #3987 > branch into tr...@head and you'll get everything from tr...@head and > everything from the branch (that's tautological, sorry, I hope you > know > what I mean). > > If you're not sure what to do with git, then here's what to do with > svn: > >BRANCHES=svn://svn.twistedmatrix.com/svn/Twisted/branches >EXPRESSIVE=$BRANCHES/expressive-http-client-886-4 >HIGHLEVEL=$BRANCHES/high-level-web-client-3987/ >cd path/to/Twisted/trunk >svn merge $express...@27272 express...@head >svn merge $highle...@27275 $highle...@head > > Or using Combinator: > >chbranch Twisted expressive-http-client-886-4 >unbranch Twisted >chbranch Twisted high-level-web-client-3987 >unbranch Twisted --force Wow. That's quite a lot of stuff. Now I have an @HEAD.ache ;-) . I'll see if I can get this whacked together in any meaningful way and maybe publish my resultant branch to Github or something. Meanwhile... ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] reverse web proxy & threads
On Mon, 2009-11-02 at 21:11 +0600, Artem Bokhan wrote:
> Hello!
>
> I'm playing with reverse proxy from examples
> (Twisted-8.2.0/doc/web/examples/proxy.py). May somebody explain what
> part of code uses threads there?
>
> pstree -p | grep python
>
> -python(12786)-+-{python}(12795)
> |-{python}(13050)
> |-{python}(15547)
> `-{python}(20767)
DNS lookups, probably: gethostbyname blocks your process, otherwise.
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
