[Twisted-Python] banana SIZE_LIMIT
Hi, Are they some simple ways (keeping existing interface) around BananaError for long values returned from PB methods? ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] more complex IBodyProducer?
has anyone implemented a more complex IBodyProducer than class POSTRequestProducer(): implements(IBodyProducer) def __init__(self, body): self.body = body self.length = len(self.body) def startProducing(self, consumer): consumer.write(self.body) return succeed(None) def pauseProducing(self): pass def stopProducing(self): pass I have an upload intensive application, that during periods of poor connectivity ends up with a lots on zombie http TCP connections. I'm thinking maybe that the "pause" is being called and that my request deferred is in a paused state. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Testing Twisted code without trial
On 22 January 2013 22:03, wrote: > On 09:29 am, a...@roiban.ro wrote: >>On 22 January 2013 02:21, wrote: >>>On 20 Jan, 02:35 pm, a...@roiban.ro wrote: I agree that this is a ugly hack and I removed the project. > > Hi Adi, > > trial does what it does by touching a lot of internal stuff as well. > This is still bad, but at least it's our fault if it ever breaks instead > of yours. There's also a long term plan (or "plan" may be putting it > too strongly, perhaps I should say "hope") that this part of trial will > change to only use public interfaces. This will probably require > reactors actually implementing restartability, or it will require > changing the trial feature slightly (eg, so it starts a reactor, runs > all tests, then stops the reactor - if it did this, I'm sure you can > imagine how "waiting" for a Deferred would just be adding a callback to > the right place, as in any other Twisted-based application). > > Are you interested in helping out with making reactors restartable? :) Sorry for the late reply. I am still clumsy when working with Twisted so I don't know if I can help to much here. I don't know what is expected from a restartable reactor. The way I am testing deferreds is by starting the reactor, allow for the deferred to execute and then stop the reactor. I don't want to pause it and then continue the execution from where it was stopped. To help with debugging I am also printing a snapshot of reactor state at a certain time. I prefer the Arrange/Act/Assert way of writing test: checker = mk.credentialsChecker() credentials = mk.credentials() deferred = checker.requestAvatarId(credentials) failure = self.getDeferredFailure(deferred) self.assertFailureType(AuthentiationError, failure) self.assertEqual(credentials.username, failure.value.username) I found it easier to read than this version: checker = mk.credentialsChecker() credentials = mk.credentials() def check_result(result_or_failure): self.assertFailureType(AuthentiationError, failure) self.assertEqual(credentials.username, failure.value.username) deferred = checker.requestAvatarId(credentials) deferred.addBoth(check_result) return deferred I have updated the code to use as many public reactor members as possible. The following private member are still use: reactor._startedBefore, reactor._started It uses the following public methods: startRunning(), doIteration(), stop(), iterate() Here is the main part that blocks the execution until the deferred got a result. It executes the deferred in the reactor loop. https://github.com/chevah/empirical/blob/master/chevah/empirical/testcase.py#L240 -- Maybe this is has only limited usage, but I just wanted to share this work. For me, this makes writing test a much nicer experience. Cheers, -- Adi Roiban ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] banana SIZE_LIMIT
On Mar 18, 2013, at 7:49 AM, Sergey Gerasimov wrote: > Hi, > > Are they some simple ways (keeping existing interface) around BananaError > for long values returned from PB methods? Don't return long values. They block the connection. Chunk them up and stream them asynchronously by repeatedly calling a method to buffer them up, and then add a .finish() method that takes the buffered data and invokes your original logic. -glyph ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] banana SIZE_LIMIT
On 07:35 pm, gl...@twistedmatrix.com wrote: > >On Mar 18, 2013, at 7:49 AM, Sergey Gerasimov wrote: >>Hi, >> >>Are they some simple ways (keeping existing interface) around >>BananaError >>for long values returned from PB methods? > >Don't return long values. They block the connection. Chunk them up >and stream them asynchronously by repeatedly calling a method to buffer >them up, and then add a .finish() method that takes the buffered data >and invokes your original logic. eg, using twisted.spread.util.Pager (or the StringPager or FilePager subclasses). Jean-Paul >-glyph > > >___ >Twisted-Python mailing list >Twisted-Python@twistedmatrix.com >http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] So... Python 3.4 is getting its own async I/O system
In Guido's keynote at PyCon 2013, apparently he talked about adding an async I/O module to Python 3.4. It looks like his slides can be viewed here: https://www.dropbox.com/s/xknbe58zcvjhzhv/PyCon2013.pptx ...while this is the PEP he's talking about: http://www.python.org/dev/peps/pep-3156/ At first glance, the proposed reactor API looks very much like Twisted's (or, to be fair, GTK's, or possibly any number of other async event loop I'm less familiar with) but rather than Deferreds and callbacks, the API will be based around Futures (similar, but not identical, to Python 3.2's concurrent.futures.Future class), and an inlineCallbacks-style decorator for generators. I know Deferreds are awesome, and I don't know much about Futures (and I know Twisted core developers have given negative reviews of other Deferred/Promise/Future implementations in, say, JavaScript libraries before), and inlineCallbacks seems to have a negative reputation among experienced Twisted users. Is there anybody on this list who knows more about this new PEP (maybe somebody who's at PyCon and saw the talk in person) who can give us an informed comparison with the current state of Twisted? ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] So... Python 3.4 is getting its own async I/O system
On Tue, Mar 19, 2013 at 11:00 AM, Tim Allen wrote: > In Guido's keynote at PyCon 2013, apparently he talked about adding an > async I/O module to Python 3.4. It looks like his slides can be viewed > here: > > https://www.dropbox.com/s/xknbe58zcvjhzhv/PyCon2013.pptx > > ...while this is the PEP he's talking about: > > http://www.python.org/dev/peps/pep-3156/ > > At first glance, the proposed reactor API looks very much like Twisted's > (or, to be fair, GTK's, or possibly any number of other async event loop > I'm less familiar with) but rather than Deferreds and callbacks, the API > will be based around Futures (similar, but not identical, to Python > 3.2's concurrent.futures.Future class), and an inlineCallbacks-style > decorator for generators. > > I know Deferreds are awesome, and I don't know much about Futures (and > I know Twisted core developers have given negative reviews of > other Deferred/Promise/Future implementations in, say, JavaScript > libraries before), and inlineCallbacks seems to have a negative > reputation among experienced Twisted users. Is there anybody on this > list who knows more about this new PEP (maybe somebody who's at PyCon > and saw the talk in person) who can give us an informed comparison with > the current state of Twisted? > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > Hi inlineCallbacks is ok to me. I use it daily. I can keep code snippets succinct. no more callbacks, deep nested callbacks (they let me recall those days with javascript & node.js). The only thing I complain is it doesn't work with cython. Regards gelin yan ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] So... Python 3.4 is getting its own async I/O system
Not relavent to your questions, but there's a mailing list for discussing the specifics of PEP 3156, which is worth subscribing to if you want to provide specific feedback on the API. https://groups.google.com/forum/?fromgroups#!forum/python-tulip On Tue, Mar 19, 2013 at 3:00 AM, Tim Allen wrote: > In Guido's keynote at PyCon 2013, apparently he talked about adding an > async I/O module to Python 3.4. It looks like his slides can be viewed > here: > > https://www.dropbox.com/s/xknbe58zcvjhzhv/PyCon2013.pptx > > ...while this is the PEP he's talking about: > > http://www.python.org/dev/peps/pep-3156/ > > At first glance, the proposed reactor API looks very much like Twisted's > (or, to be fair, GTK's, or possibly any number of other async event loop > I'm less familiar with) but rather than Deferreds and callbacks, the API > will be based around Futures (similar, but not identical, to Python > 3.2's concurrent.futures.Future class), and an inlineCallbacks-style > decorator for generators. > > I know Deferreds are awesome, and I don't know much about Futures (and > I know Twisted core developers have given negative reviews of > other Deferred/Promise/Future implementations in, say, JavaScript > libraries before), and inlineCallbacks seems to have a negative > reputation among experienced Twisted users. Is there anybody on this > list who knows more about this new PEP (maybe somebody who's at PyCon > and saw the talk in person) who can give us an informed comparison with > the current state of Twisted? > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
