[Twisted-Python] util.quote deprecated
Hi, what is a proper way to escape user input in database query strings? I've used quote from twisted.enterprise.util, but it is deprecated now. Is there any other module for this purpose? Thanks, Pet ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] util.quote deprecated
On Tue, Mar 3, 2009 at 2:04 PM, Tim Allen wrote:
> On Tue, Mar 03, 2009 at 01:17:48PM +0100, Pet wrote:
> > what is a proper way to escape user input in database query strings?
> > I've used quote from twisted.enterprise.util, but it is deprecated now.
> > Is there any other module for this purpose?
>
> The correct way to escape user input is not to do it at all, but rather
> to leave it up to the DB-API module you're using:
>
>from twisted.enterprise.adbapi import ConnectionPool
>
>pool = ConnectionPool("psycopg2")
>d = pool.runQuery("""
>SELECT *
>FROM students
>WHERE name = %s
>""", "Robert '); DROP TABLE students;--")
>
> Note that although I've used "%s" in the query, this is not normal
> Python string-formatting, the "%s" is just tells the DB-API module I'm
> using (in this case, psycopg2 for PostgreSQL) to quote one of the extra
> parameters and insert in that spot. Look up "paramstyle" in the DB-API
> spec[1] and the documentation for the DB-API module you're using for
> more details.
>
> [1] http://www.python.org/dev/peps/pep-0249/
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
Thanks for all your answers! It works fine in that way. But what, if I
compose my query? For example:
def getData(self, type=''):
id = 1
if type:
str = " AND mytype = %s " % type
str = ''
query = "SELECT * FROM table WHERE id = %s %s " % (id,str)
cur.execute(query)
I mean, str part is not always there and I need escape it only if type is
passed to function
Thanks, Pet
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] twistd tap file are not created
Hello,
I'm trying an example from twisted book and when I run this:
twistd -n reverse_app.py
application works, but no .tap file is created
Another question, how are twisted daemons actually stopped?
Pet
reverse_app.py
from twisted.application import service
import reverse
application = service.Application("Reverser")
reverserService = reverse.ReverserService()
reverserService.setServiceParent(application)
reverse.py:
from twisted.application import service, internet
from twisted.internet import protocol, reactor
from twisted.protocols import basic
def reverse(string):
return string[::-1]
class ReverserProtocol(basic.LineReceiver):
def lineReceived(self, line):
if hasattr(self, 'handle_' + line):
getattr(self, 'handle_' + line)()
else:
self.sendLine(reverse(line))
def handle_quit(self):
self.transport.loseConnection()
class ReverserFactory(protocol.ServerFactory):
protocol = ReverserProtocol
class ReverserService(internet.TCPServer):
def _ _init_ _(self):
internet.TCPServer.__init__(self, 2323, ReverserFactory())
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] twistd tap file are not created
On Mon, Mar 9, 2009 at 9:02 PM, Drew Smathers wrote: > On Mon, Mar 9, 2009 at 8:02 AM, Pet wrote: >> Hello, >> >> I'm trying an example from twisted book and when I run this: >> twistd -n reverse_app.py >> >> application works, but no .tap file is created >> > > Are you sure the book states you can build tap files with twistd? Yes > mktap is the program for that. Nonetheless, .tap files are > deprecated: I don't really need them, at least by now. I've just wanted be sure everything works fine > > http://twistedmatrix.com/projects/core/documentation/howto/tap.html#auto1 > >> Another question, how are twisted daemons actually stopped? >> > > With the UNIX kill command ;) > > $ kill `cat twisted.pid` > > Until this (http://twistedmatrix.com/trac/ticket/823) is resolved. I've wrote little shell script for that. But it doesn't check if commands are successfully executed or if app already running (status). Anyway it works for me in that simple way. > > -Drew > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] LineReceiver MAX_LENGTH limitation
Hi, in LineReceiver there is MAX_LENGTH = 16384 limitation. Is it possible to set it bigger at start of an application? Regards, Pet ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] LineReceiver MAX_LENGTH limitation
On Wed, Mar 18, 2009 at 1:19 PM, Jean-Paul Calderone wrote: > On Wed, 18 Mar 2009 13:10:12 +0100, Pet wrote: >> >> Hi, >> >> in LineReceiver there is MAX_LENGTH = 16384 limitation. Is it possible >> to set it bigger at start of an application? > > It's an attribute. Not only can you set it to a larger value at the start > of an application, but you can also set it at the middle or the end of an > application. ;) Ups! I've tried and it didn't work for probably another reason. Thanks, it works fine now. :) Pet > > Jean-Paul > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] db connections
Hi,
I've wrote an daemon which does some queries to db and sends response
back to client. Do I need to make for every request from client (new
instance of MyProtocol) a new connection to DB? Or can I somehow
prepare connection, so I could save some time? Maybe make connection
in Factory and pass it to Protocol? But what happens if too much
clients are connected to server? What is the way to control it?
Regards, Pet
class MyProtocol(basic.LineReceiver):
def __init__(self):
print "new connection"
self.dbcon = adbapi.ConnectionPool("pyPgSQL.PgSQL",
database="data", user='pet', host='local', password='some')
class MyFactory(protocol.ServerFactory):
protocol = MyProtocol
class MyService(internet.TCPServer):
def __init__(self):
internet.TCPServer.__init__(self,PORT,MyFactory())
def main():
reactor.listenTCP(PORT, MyFactory())
reactor.run()
if __name__ == '__main__':
main()
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] db connections
On Tue, Mar 24, 2009 at 2:45 PM, Alvin Delagon wrote:
> Put self.dbcon in the MyFactory class. MyProtocol instances can access it
> via self.factory.
Thanks for your help! Could you give me an example? I'm getting an error
MyProtocol instance has no attribute 'factory'
if I do as you suggested.
>
>
> On Tue, Mar 24, 2009 at 6:10 PM, Pet wrote:
>>
>> Hi,
>>
>> I've wrote an daemon which does some queries to db and sends response
>> back to client. Do I need to make for every request from client (new
>> instance of MyProtocol) a new connection to DB? Or can I somehow
>> prepare connection, so I could save some time? Maybe make connection
>> in Factory and pass it to Protocol? But what happens if too much
>> clients are connected to server? What is the way to control it?
>>
>> Regards, Pet
>>
>>
>> class MyProtocol(basic.LineReceiver):
>>
>> def __init__(self):
>> print "new connection"
>> self.dbcon = adbapi.ConnectionPool("pyPgSQL.PgSQL",
>> database="data", user='pet', host='local', password='some')
>>
>>
>> class MyFactory(protocol.ServerFactory):
>> protocol = MyProtocol
>>
>> class MyService(internet.TCPServer):
>> def __init__(self):
>> internet.TCPServer.__init__(self,PORT,MyFactory())
>>
>> def main():
>> reactor.listenTCP(PORT, MyFactory())
>> reactor.run()
>>
>>
>> if __name__ == '__main__':
>> main()
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
>
> --
> http://www.alvinatorsplayground.blogspot.com/
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] db connections
On Tue, Mar 24, 2009 at 5:04 PM, Rob Hoadley wrote:
> You'd want to use a connection pool to manage the db interaction.
> Your server is pretty unusable after a db connection failure. I've
> used the connection pool before with a cp_min of 1 and a cp_max of 2.
Honestly speaking, I don't understand, what does it mean.
I'm already using connection pool with default cp_min an max, don't I?
Could you explain me, as for a newbie, please
>
> http://twistedmatrix.com/documents/8.2.0/api/twisted.enterprise.adbapi.ConnectionPool.html
>
>
> On Tue, Mar 24, 2009 at 6:45 AM, Alvin Delagon wrote:
>> Put self.dbcon in the MyFactory class. MyProtocol instances can access it
>> via self.factory.
>>
>>
>> On Tue, Mar 24, 2009 at 6:10 PM, Pet wrote:
>>>
>>> Hi,
>>>
>>> I've wrote an daemon which does some queries to db and sends response
>>> back to client. Do I need to make for every request from client (new
>>> instance of MyProtocol) a new connection to DB? Or can I somehow
>>> prepare connection, so I could save some time? Maybe make connection
>>> in Factory and pass it to Protocol? But what happens if too much
>>> clients are connected to server? What is the way to control it?
>>>
>>> Regards, Pet
>>>
>>>
>>> class MyProtocol(basic.LineReceiver):
>>>
>>> def __init__(self):
>>> print "new connection"
>>> self.dbcon = adbapi.ConnectionPool("pyPgSQL.PgSQL",
>>> database="data", user='pet', host='local', password='some')
>>>
>>>
>>> class MyFactory(protocol.ServerFactory):
>>> protocol = MyProtocol
>>>
>>> class MyService(internet.TCPServer):
>>> def __init__(self):
>>> internet.TCPServer.__init__(self,PORT,MyFactory())
>>>
>>> def main():
>>> reactor.listenTCP(PORT, MyFactory())
>>> reactor.run()
>>>
>>>
>>> if __name__ == '__main__':
>>> main()
>>>
>>> ___
>>> Twisted-Python mailing list
>>> Twisted-Python@twistedmatrix.com
>>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>>
>>
>> --
>> http://www.alvinatorsplayground.blogspot.com/
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] db connections
On Tue, Mar 24, 2009 at 6:10 PM, Alvin Delagon wrote:
> Something like this:
>
> from twisted.protocols import basic
> from twisted.internet import protocol, reactor
> from twisted.enterprise import adbapi
>
> class MyProtocol(basic.LineReceiver):
> def __init__(self):
> pass
>
> def lineReceived(self, line):
> ### dbcon can be accessed via self.factory
> print dir(self.factory.dbcon)
>
> class MyFactory(protocol.ServerFactory):
> protocol = MyProtocol
> def __init__(self):
> self.dbcon = adbapi.ConnectionPool("pyPgSQL.PgSQL", database="data",
> user='pet', host='local', password='some')
>
> if __name__ == "__main__":
> reactor.listenTCP(8080, MyFactory())
> reactor.run()
>
> I usually put persistent data on the factory so that protocol instances can
> access them such as {username:} dictionary for chatroom
> server.
Hi,
thanks for example!
In that way, I'm getting error:
exceptions.AttributeError: MyProtocol instance has no attribute 'factory'
How can MyProtocol access self.factory.dbcon?
Pet
>
>
>
> On Wed, Mar 25, 2009 at 12:22 AM, Pet wrote:
>>
>> On Tue, Mar 24, 2009 at 5:04 PM, Rob Hoadley wrote:
>> > You'd want to use a connection pool to manage the db interaction.
>> > Your server is pretty unusable after a db connection failure. I've
>> > used the connection pool before with a cp_min of 1 and a cp_max of 2.
>>
>> Honestly speaking, I don't understand, what does it mean.
>> I'm already using connection pool with default cp_min an max, don't I?
>> Could you explain me, as for a newbie, please
>>
>> >
>> >
>> > http://twistedmatrix.com/documents/8.2.0/api/twisted.enterprise.adbapi.ConnectionPool.html
>> >
>> >
>> > On Tue, Mar 24, 2009 at 6:45 AM, Alvin Delagon
>> > wrote:
>> >> Put self.dbcon in the MyFactory class. MyProtocol instances can access
>> >> it
>> >> via self.factory.
>> >>
>> >>
>> >> On Tue, Mar 24, 2009 at 6:10 PM, Pet wrote:
>> >>>
>> >>> Hi,
>> >>>
>> >>> I've wrote an daemon which does some queries to db and sends response
>> >>> back to client. Do I need to make for every request from client (new
>> >>> instance of MyProtocol) a new connection to DB? Or can I somehow
>> >>> prepare connection, so I could save some time? Maybe make connection
>> >>> in Factory and pass it to Protocol? But what happens if too much
>> >>> clients are connected to server? What is the way to control it?
>> >>>
>> >>> Regards, Pet
>> >>>
>> >>>
>> >>> class MyProtocol(basic.LineReceiver):
>> >>>
>> >>> def __init__(self):
>> >>> print "new connection"
>> >>> self.dbcon = adbapi.ConnectionPool("pyPgSQL.PgSQL",
>> >>> database="data", user='pet', host='local', password='some')
>> >>>
>> >>>
>> >>> class MyFactory(protocol.ServerFactory):
>> >>> protocol = MyProtocol
>> >>>
>> >>> class MyService(internet.TCPServer):
>> >>> def __init__(self):
>> >>> internet.TCPServer.__init__(self,PORT,MyFactory())
>> >>>
>> >>> def main():
>> >>> reactor.listenTCP(PORT, MyFactory())
>> >>> reactor.run()
>> >>>
>> >>>
>> >>> if __name__ == '__main__':
>> >>> main()
>> >>>
>> >>> ___
>> >>> Twisted-Python mailing list
>> >>> Twisted-Python@twistedmatrix.com
>> >>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>> >>
>> >>
>> >>
>> >> --
>> >> http://www.alvinatorsplayground.blogspot.com/
>> >>
>> >> ___
>> >> Twisted-Python mailing list
>> >> Twisted-Python@twistedmatrix.com
>> >> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>> >>
>> >>
>> >
>> > ___
>> > Twisted-Python mailing list
>> > Twisted-Python@twistedmatrix.com
>> > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>> >
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
>
> --
> http://www.alvinatorsplayground.blogspot.com/
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] db connections
On Wed, Mar 25, 2009 at 10:22 AM, Pet wrote:
> On Tue, Mar 24, 2009 at 6:10 PM, Alvin Delagon wrote:
>> Something like this:
>>
>> from twisted.protocols import basic
>> from twisted.internet import protocol, reactor
>> from twisted.enterprise import adbapi
>>
>> class MyProtocol(basic.LineReceiver):
>> def __init__(self):
>> pass
>>
>> def lineReceived(self, line):
>> ### dbcon can be accessed via self.factory
>> print dir(self.factory.dbcon)
>>
>> class MyFactory(protocol.ServerFactory):
>> protocol = MyProtocol
>> def __init__(self):
>> self.dbcon = adbapi.ConnectionPool("pyPgSQL.PgSQL", database="data",
>> user='pet', host='local', password='some')
>>
>> if __name__ == "__main__":
>> reactor.listenTCP(8080, MyFactory())
>> reactor.run()
>>
>> I usually put persistent data on the factory so that protocol instances can
>> access them such as {username:} dictionary for chatroom
>> server.
>
> Hi,
>
> thanks for example!
> In that way, I'm getting error:
>
> exceptions.AttributeError: MyProtocol instance has no attribute 'factory'
>
> How can MyProtocol access self.factory.dbcon?
Ups! I didn't followed exactly your example and tried to access
self.factory.dbcon in __init__ of MyProtocol, so I've got an error.
If I access self.factory.dbcon in lineReceived it seems to work. But I
don't really understand, why can I access factory in lineReceived and
not in __init__.
Pet
>
>
> Pet
>
>
>
>
>>
>>
>>
>> On Wed, Mar 25, 2009 at 12:22 AM, Pet wrote:
>>>
>>> On Tue, Mar 24, 2009 at 5:04 PM, Rob Hoadley wrote:
>>> > You'd want to use a connection pool to manage the db interaction.
>>> > Your server is pretty unusable after a db connection failure. I've
>>> > used the connection pool before with a cp_min of 1 and a cp_max of 2.
>>>
>>> Honestly speaking, I don't understand, what does it mean.
>>> I'm already using connection pool with default cp_min an max, don't I?
>>> Could you explain me, as for a newbie, please
>>>
>>> >
>>> >
>>> > http://twistedmatrix.com/documents/8.2.0/api/twisted.enterprise.adbapi.ConnectionPool.html
>>> >
>>> >
>>> > On Tue, Mar 24, 2009 at 6:45 AM, Alvin Delagon
>>> > wrote:
>>> >> Put self.dbcon in the MyFactory class. MyProtocol instances can access
>>> >> it
>>> >> via self.factory.
>>> >>
>>> >>
>>> >> On Tue, Mar 24, 2009 at 6:10 PM, Pet wrote:
>>> >>>
>>> >>> Hi,
>>> >>>
>>> >>> I've wrote an daemon which does some queries to db and sends response
>>> >>> back to client. Do I need to make for every request from client (new
>>> >>> instance of MyProtocol) a new connection to DB? Or can I somehow
>>> >>> prepare connection, so I could save some time? Maybe make connection
>>> >>> in Factory and pass it to Protocol? But what happens if too much
>>> >>> clients are connected to server? What is the way to control it?
>>> >>>
>>> >>> Regards, Pet
>>> >>>
>>> >>>
>>> >>> class MyProtocol(basic.LineReceiver):
>>> >>>
>>> >>> def __init__(self):
>>> >>> print "new connection"
>>> >>> self.dbcon = adbapi.ConnectionPool("pyPgSQL.PgSQL",
>>> >>> database="data", user='pet', host='local', password='some')
>>> >>>
>>> >>>
>>> >>> class MyFactory(protocol.ServerFactory):
>>> >>> protocol = MyProtocol
>>> >>>
>>> >>> class MyService(internet.TCPServer):
>>> >>> def __init__(self):
>>> >>> internet.TCPServer.__init__(self,PORT,MyFactory())
>>> >>>
>>> >>> def main():
>>> >>> reactor.listenTCP(PORT, MyFactory())
>>> >>> reactor.run()
>>> >>>
>>> >>>
>>> >>> if __name__ == '__main__':
>>> >>> main()
>>> >>>
>>> >>> ___
>>> >>> Twisted-Python mailing list
>>> >>> Twisted-Python@twistedmatrix.com
>>> >>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >> http://www.alvinatorsplayground.blogspot.com/
>>> >>
>>> >> ___
>>> >> Twisted-Python mailing list
>>> >> Twisted-Python@twistedmatrix.com
>>> >> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>> >>
>>> >>
>>> >
>>> > ___
>>> > Twisted-Python mailing list
>>> > Twisted-Python@twistedmatrix.com
>>> > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>> >
>>>
>>> ___
>>> Twisted-Python mailing list
>>> Twisted-Python@twistedmatrix.com
>>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>>
>>
>> --
>> http://www.alvinatorsplayground.blogspot.com/
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] db connections
On Sat, Mar 28, 2009 at 2:32 AM, Enrique Samson Jr. wrote: > On Wed, 2009-03-25 at 11:00 +0100, Pet wrote: >> > Hi, >> > >> > thanks for example! >> > In that way, I'm getting error: >> > >> > exceptions.AttributeError: MyProtocol instance has no attribute 'factory' >> > >> > How can MyProtocol access self.factory.dbcon? >> >> Ups! I didn't followed exactly your example and tried to access >> self.factory.dbcon in __init__ of MyProtocol, so I've got an error. >> If I access self.factory.dbcon in lineReceived it seems to work. But I >> don't really understand, why can I access factory in lineReceived and >> not in __init__. >> >> Pet > > Hi Pet, > > You can simply look into the buildProtocol code of Factory for an > explanation. This is the part where your protocol gets to be > instantiated. thanks for a hint. I'll look into that closer. Twisted is like a black box for me right now. However, I've managed to write a useful Client-Server App :) > > > HTH, > > Enrique > > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] design question
Hi,
I've written my first Twisted Server, which accept request from
clients, does DB queries, sends response back and logs some events in
DB. I wonder, if my design was good.
Basically, I have a Factory where is db connectionPool created and
Protocol(basic.LineReceiver), which processes requests and sends data
back.
In Protocol, I instantiate my utility object which I try to keep
independent from twisted framework, so it can be easily used
somewhere else or can be replaced by other class.
def m(self, *args, **kwargs ):
try:
d = self.getData(params=kwargs['params'])
if d:
d.addCallback(self.sendResult)
d.addErrback(log.err)
d.addCallback(self.logFailedQuery)
d.addErrback(log.err)
return None
except Exception, e:
log.msg('Unknown error in m:', e)
def getData(self, *args, **kwargs):
obj = MyObj(self.db)
return self.db.runInteraction(obj.getDataFromDB,kwargs['parameter'])
What I don't like, I must pass db variable to obj.getDataFromDb and
may be other function called there. I think it would be better obj =
MyObj(self.db), but it doesn't work and runInteraction passes
automatically db variable to interaction method.
I'm new to Python and Twisted and would appreciate your advises.
Pet
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] XML-RPC for paralell clients
Hi, after playing with reactor, factory and protocol, I'm trying to implement XML-RPC server, but it seems that it is not able to handle several clients in parallel. If I call the slow function with first client and then with second client a fast function, second client will receiver the results after slow function returned. There is no factory for clients in server.Site() like in Protokoll? What is the way to do that for XML-RPC? Thanks for your help! Pet from twisted.web import xmlrpc, server from twisted.application import internet from time import sleep PORT = 1 class ExampleXMLRPCProtokoll(xmlrpc.XMLRPC): """An example object to be published.""" def xmlrpc_echo(self, x): """ Return all passed args. """ return x def xmlrpc_add(self, a, b): """ Return sum of arguments. """ return a + b def xmlrpc_addslow(self, a, b): """ Return sum of arguments. takes time """ sleep(20) return a + b def xmlrpc_fault(self): """ Raise a Fault indicating that the procedure should not be used. """ raise xmlrpc.Fault(123, "The fault procedure is faulty.") class ExampleXMLRPCService(internet.TCPServer): def __init__(self): print "Starting XML-RPC Service..." r = ExampleXMLRPCProtokoll() internet.TCPServer.__init__(self,PORT,server.Site(r)) def main(): from twisted.internet import reactor r = ExampleXMLRPCProtokoll() reactor.listenTCP(PORT, server.Site(r)) reactor.run() if __name__ == '__main__': main() ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] XML-RPC for paralell clients
On Tue, May 19, 2009 at 10:59 AM, wrote: > On 07:51 am, petshm...@googlemail.com wrote: >>after playing with reactor, factory and protocol, I'm trying to >>implement XML-RPC server, but it seems that it is not able to handle >>several clients in parallel. If I call the slow function with first >>client and then with second client a fast function, second client >>will receiver the results after slow function returned. There is no >>factory for clients in server.Site() like in Protokoll? What is the >>way to do that for XML-RPC? > > Whenever you perform some blocking work in a Twisted program, the > reactor will block and no processing will occur in parallel. This is > not specific to XML-RPC. server.Site() *is* a Factory for clients, > exactly the same as any other protocol. You can see its place in the > inheritance hierarchy: Thank you for explanations! I still have some questions :( if server.Site() is Factory clients, what object is created for each connection? How can I ensure, that each client gets answer belonging to him and not for other "slow" client? How can I close connection to a client? > > http://twistedmatrix.com/documents/8.2.0/api/classIndex.html#twisted.web.server.Site >> def xmlrpc_addslow(self, a, b): >> """ >> Return sum of arguments. takes time >> """ >> sleep(20) >> return a + b > > The Twisted equivalent way to spell this is to return a Deferred which > fires later from your xmlrpc_addslow method: > > from twisted.internet.task import deferLater > from twisted.internet import reactor > # ... > def xmlrpc_addslow(self, a, b): > return deferLater(reactor, 20, lambda : a + b) > > However, if your "sleep(20)" is intended to simulate 20 seconds of work > in a blocking API you can't control, you might want to return a Deferred > that does the work in a thread instead: > > from time import sleep > from twisted.internet.threads import deferToThread > # ... > def xmlrpc_addslow(self, a, b): > def hardWork(): > sleep(20) > return a + b > return deferToThread(hardWork) this should be ok for me. > > As with all Python programs, if you truly want CPU parallelism, then you > will need to put the work into a subprocess instead. You might want to > look at the "ampoule" process-pool project: > https://launchpad.net/ampoule > > For many applications though, you shouldn't need to worry about > parallelism this much. Since Twisted's XMLRPC server support allows you > to replace any method's return value with a Deferred without changing > the externally observable behavior of XML-RPC server, you can add > parallelism incrementally as you actually require it. In other words, > you probably don't need to worry about it as much as you think :). > > Hope this helped! > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] XML-RPC for paralell clients
On Tue, May 19, 2009 at 11:48 AM, Pet wrote: > On Tue, May 19, 2009 at 10:59 AM, wrote: >> On 07:51 am, petshm...@googlemail.com wrote: >>>after playing with reactor, factory and protocol, I'm trying to >>>implement XML-RPC server, but it seems that it is not able to handle >>>several clients in parallel. If I call the slow function with first >>>client and then with second client a fast function, second client >>>will receiver the results after slow function returned. There is no >>>factory for clients in server.Site() like in Protokoll? What is the >>>way to do that for XML-RPC? >> >> Whenever you perform some blocking work in a Twisted program, the >> reactor will block and no processing will occur in parallel. This is >> not specific to XML-RPC. server.Site() *is* a Factory for clients, >> exactly the same as any other protocol. You can see its place in the >> inheritance hierarchy: > > Thank you for explanations! > > I still have some questions :( > if server.Site() is Factory clients, what object is created for each > connection? > How can I ensure, that each client gets answer belonging to him and > not for other "slow" client? How can I close connection to a client? deferToThread or db.runInteraction seems to be the solution for me. The "only" thing I don't know how to do is, logging requests *after* response was send to the client. Obviously, it should happens after a xmlrpc_method returns, but how can accomplish this? Thanks for any help! > > >> >> http://twistedmatrix.com/documents/8.2.0/api/classIndex.html#twisted.web.server.Site >>> def xmlrpc_addslow(self, a, b): >>> """ >>> Return sum of arguments. takes time >>> """ >>> sleep(20) >>> return a + b >> >> The Twisted equivalent way to spell this is to return a Deferred which >> fires later from your xmlrpc_addslow method: >> >> from twisted.internet.task import deferLater >> from twisted.internet import reactor >> # ... >> def xmlrpc_addslow(self, a, b): >> return deferLater(reactor, 20, lambda : a + b) >> >> However, if your "sleep(20)" is intended to simulate 20 seconds of work >> in a blocking API you can't control, you might want to return a Deferred >> that does the work in a thread instead: >> >> from time import sleep >> from twisted.internet.threads import deferToThread >> # ... >> def xmlrpc_addslow(self, a, b): >> def hardWork(): >> sleep(20) >> return a + b >> return deferToThread(hardWork) > > this should be ok for me. > >> >> As with all Python programs, if you truly want CPU parallelism, then you >> will need to put the work into a subprocess instead. You might want to >> look at the "ampoule" process-pool project: >> https://launchpad.net/ampoule >> >> For many applications though, you shouldn't need to worry about >> parallelism this much. Since Twisted's XMLRPC server support allows you >> to replace any method's return value with a Deferred without changing >> the externally observable behavior of XML-RPC server, you can add >> parallelism incrementally as you actually require it. In other words, >> you probably don't need to worry about it as much as you think :). >> >> Hope this helped! >> >> ___ >> Twisted-Python mailing list >> Twisted-Python@twistedmatrix.com >> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python >> > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] can addCallback accept list of deferreds?
Hi,
I'm trying to process several items in callback function writeLog
d.addCallback(self.writeLog,g=g)
see below.
Can writeLog return DeferredList instead of Deferred?
WriteLog should run several inserts into db (some of which may fail).
Or am I completely wrong here?
Thanks, Pet
def getData(self, *args, **kwargs ):
try:
g = MyClass(self.db)
d = self.getResult(params=kwargs['params'], g=g)
if d:
d.addCallback(self.sendResult)
d.addErrback(log.err)
d.addCallback(self.writeLog,g=g)
d.addErrback(log.err)
return d
except Exception, e:
log.msg('Unknown error in getData:', e)
return False
def writeLog(self, *args, **kwargs):
g = kwargs['g']
dlist = [self.wLog(g, p) for p in g.list]
d = DeferredList(dlist, consumeErrors=True).addCallback(log.err)
return d
def wLog(self, g, p):
return self.db.runInteraction(g.insertLog,p)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] unicode for pyPgSQL
Hello, does pyPgSQL doesn't support unicode? execute(query, params) #all items in params list are of I'm getting 'ascii' codec can't encode character u'\xfc' in position 1: ordinal not in range(128) Do I need encode my params before sending a query and then decode all results back to unicode? Pet ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] unicode for pyPgSQL
On Wed, Aug 12, 2009 at 6:51 PM, wrote: > On 03:28 pm, petshm...@googlemail.com wrote: >>Hello, >> >>does pyPgSQL doesn't support unicode? >> >>execute(query, params) #all items in params list are of >'unicode'> >> >>I'm getting >>'ascii' codec can't encode character u'\xfc' in position 1: ordinal >>not in range(128) >> >>Do I need encode my params before sending a query and then decode all >>results back to unicode? > > I don't know about pyPgSQL's unicode support. > > I did recently learn that pyPgSQL doesn't support bind parameters, > apparently resulting in almost any use of it insecure and vulnerable to > SQL injection attacks. You may want to investigate this further before > deciding if it's worth figuring out how unicode works. That's interesting. May be I should try psycorg2. By the way, pyPgSQL has no problem with unicode, the problem was elsewhere > > Jean-Paul > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] run queries in deffered, but not in transaction
Hi, I'd like to run several queries in background, some of them may fail. Currently, I do runInteraction for each query. Is there a way to run them all in one deferred, so that if one of them fails, other are not aborted? Thanks in advance! Pet ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] run queries in deffered, but not in transaction
On Tue, Sep 15, 2009 at 5:19 PM, Mark Visser wrote: > exar...@twistedmatrix.com wrote: >> On 10:37 am, petshm...@googlemail.com wrote: >>> >>> I'd like to run several queries in background, some of them may fail. >>> >> >> If you have a function along the lines of this one: >> >> def someInteractions(db): >> interactions = [ >> db.runInteraction(one), >> db.runInteraction(two), >> db.runInteraction(three)] >> >> Then a failure in one shouldn't affect two or three; likewise for any >> other failure or combination of failures. They are naturally (ugh, not >> a good word, but I can't think of a better one) independent. You have >> to go out of your way to associate them somehow. >> > I think he might mean he wants them to run sequentially, even if one fails. > > You can do that explicitly via @inlineCallbacks like this: > > @inlineCallbacks > def someInteractions(db): > try: > yield db.runInteraction(one) > except: > pass > > try: > yield db.runInteraction(two) > except: > pass > > try: > yield db.runInteraction(three) > except: > pass > > Or with callback/errbacks, like this: > > def someInteractions(db) > d = db.runInteraction(one).addBoth(db.runInteraction, > two).addBoth(db.runInteraction, three) Hi Mark! Yes, I'd like run them sequentially, it was not clear for me, how to do it in one deferred. I will try your suggestions out. Thanks for help! Pet > > addBoth is a convenience method that adds the same function as a > callback and an errback: > > http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.defer.Deferred.html#addBoth > > -- > Mark Visser, Software Director > Lumière VFX > Email: ma...@lumierevfx.com > Phone: +1-514-316-1080 x3030 > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] run queries in deffered, but not in transaction
On Tue, Sep 15, 2009 at 6:21 PM, Pet wrote: > On Tue, Sep 15, 2009 at 5:19 PM, Mark Visser wrote: >> exar...@twistedmatrix.com wrote: >>> On 10:37 am, petshm...@googlemail.com wrote: >>>> >>>> I'd like to run several queries in background, some of them may fail. >>>> >>> >>> If you have a function along the lines of this one: >>> >>> def someInteractions(db): >>> interactions = [ >>> db.runInteraction(one), >>> db.runInteraction(two), >>> db.runInteraction(three)] >>> >>> Then a failure in one shouldn't affect two or three; likewise for any >>> other failure or combination of failures. They are naturally (ugh, not >>> a good word, but I can't think of a better one) independent. You have >>> to go out of your way to associate them somehow. >>> >> I think he might mean he wants them to run sequentially, even if one fails. >> >> You can do that explicitly via @inlineCallbacks like this: >> >> @inlineCallbacks >> def someInteractions(db): >> try: >> yield db.runInteraction(one) >> except: >> pass >> >> try: >> yield db.runInteraction(two) >> except: >> pass >> >> try: >> yield db.runInteraction(three) >> except: >> pass >> >> Or with callback/errbacks, like this: >> >> def someInteractions(db) >> d = db.runInteraction(one).addBoth(db.runInteraction, >> two).addBoth(db.runInteraction, three) > Hi, I've tried to do following: def logRequest(self, *arg, **kw): obj = copy.deepcopy(kw['obj']) d = self.db.runInteraction(obj.first) d.addCallback(self.db.runInteraction, obj.second, param1, param2) d.addErrback(log.err) d.addCallback(self.db.runInteraction, obj.third) d.addErrback(log.err) unfortunately it doesn't work in that way, because I suppose, obj is destroyed if second or third interaction starts. Is there a way to solve this? Thanks, Pet > Hi Mark! > > Yes, I'd like run them sequentially, it was not clear for me, how to > do it in one deferred. > > I will try your suggestions out. > > Thanks for help! > > Pet > >> >> addBoth is a convenience method that adds the same function as a >> callback and an errback: >> >> http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.defer.Deferred.html#addBoth >> >> -- >> Mark Visser, Software Director >> Lumière VFX >> Email: ma...@lumierevfx.com >> Phone: +1-514-316-1080 x3030 >> >> >> ___ >> Twisted-Python mailing list >> Twisted-Python@twistedmatrix.com >> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python >> > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] run queries in deffered, but not in transaction
On Thu, Sep 17, 2009 at 4:35 AM, Matt Perry wrote: > Make sure you're using the @inlineCallbacks decorator and the yield > statement referenced previously. Without those you're just adding several > callbacks to the same Deferred; with them, the function will wait until the > Deferred fires before continuing. > > def logRequest(self, *arg, **kw): > obj = copy.deepcopy(kw['obj']) > > d = self.db.runInteraction(obj.first) > > d.addCallback(self.db.runInteraction, obj.second, param1, param2) > d.addErrback(log.err) > > d.addCallback(self.db.runInteraction, obj.third) > d.addErrback(log.err) > > > Note that "d" is the same Deferred throughout. If you change it to: > > @inlineCallbacks > def logRequest(self, *arg, **kw): > obj = copy.deepcopy(kw['obj']) > > firstDeferred = self.db.runInteraction(obj.first) > yield firstDeferred > > # code here won't get resumed until "firstDeferred" has a result > print firstDeferred.result > > secondDeferred = self.db.runInteraction(obj.second) > yield secondDeferred > > # same as above, code won't get executed until "secondDeferred" has a > result > > then you'll get the behavior you're looking for. > > See > http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.defer.html#inlineCallbacks > for some more information. > Matt and Jean-Paul, thank you for help! Pet > - Matt > > > > On Wed, Sep 16, 2009 at 12:18 PM, Pet wrote: >> >> On Tue, Sep 15, 2009 at 6:21 PM, Pet wrote: >> > On Tue, Sep 15, 2009 at 5:19 PM, Mark Visser >> > wrote: >> >> exar...@twistedmatrix.com wrote: >> >>> On 10:37 am, petshm...@googlemail.com wrote: >> >>>> >> >>>> I'd like to run several queries in background, some of them may fail. >> >>>> >> >>> >> >>> If you have a function along the lines of this one: >> >>> >> >>> def someInteractions(db): >> >>> interactions = [ >> >>> db.runInteraction(one), >> >>> db.runInteraction(two), >> >>> db.runInteraction(three)] >> >>> >> >>> Then a failure in one shouldn't affect two or three; likewise for any >> >>> other failure or combination of failures. They are naturally (ugh, >> >>> not >> >>> a good word, but I can't think of a better one) independent. You have >> >>> to go out of your way to associate them somehow. >> >>> >> >> I think he might mean he wants them to run sequentially, even if one >> >> fails. >> >> >> >> You can do that explicitly via @inlineCallbacks like this: >> >> >> >> @inlineCallbacks >> >> def someInteractions(db): >> >> try: >> >> yield db.runInteraction(one) >> >> except: >> >> pass >> >> >> >> try: >> >> yield db.runInteraction(two) >> >> except: >> >> pass >> >> >> >> try: >> >> yield db.runInteraction(three) >> >> except: >> >> pass >> >> >> >> Or with callback/errbacks, like this: >> >> >> >> def someInteractions(db) >> >> d = db.runInteraction(one).addBoth(db.runInteraction, >> >> two).addBoth(db.runInteraction, three) >> > >> >> Hi, >> >> >> I've tried to do following: >> >> def logRequest(self, *arg, **kw): >> obj = copy.deepcopy(kw['obj']) >> >> d = self.db.runInteraction(obj.first) >> >> d.addCallback(self.db.runInteraction, obj.second, param1, param2) >> d.addErrback(log.err) >> >> d.addCallback(self.db.runInteraction, obj.third) >> d.addErrback(log.err) >> >> >> unfortunately it doesn't work in that way, because I suppose, obj is >> destroyed if second or third interaction starts. >> Is there a way to solve this? >> >> Thanks, Pet >> >> > Hi Mark! >> > >> > Yes, I'd like run them sequentially, it was not clear for me, how to >> > do it in one deferred. >> > >> > I will try your suggestions out. >> > >> > Thanks for help! >> > >> > Pet >> > >> >> >> >> addBoth is a convenience method that adds the same function as a >> >> callback and an errback: >> >> >> >> >> >> http://twistedmatrix.com/documents/8.2.0/api/twisted.internet.defer.Deferred.html#addBoth >> >> >> >> -- >> >> Mark Visser, Software Director >> >> Lumière VFX >> >> Email: ma...@lumierevfx.com >> >> Phone: +1-514-316-1080 x3030 >> >> >> >> >> >> ___ >> >> Twisted-Python mailing list >> >> Twisted-Python@twistedmatrix.com >> >> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python >> >> >> > >> >> ___ >> Twisted-Python mailing list >> Twisted-Python@twistedmatrix.com >> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] inlineCallbacks
Hi,
calling in client code self.cred.login() I can't print login result.
What I'm doing wrong? login method returns deffered, but should yield
result from callRemote method.
Thanks for any help!
Pet
class Cred:
@inlineCallbacks
def login(self):
proxy = Proxy(LOGIN_PROXY)
l = {"user":LOGIN_USERNAME, "pass":LOGIN_PASSWD}
loginResult = yield proxy.callRemote('login', l)
print "RESULT", loginResult.result
returnValue(loginResult)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] inlineCallbacks
On Thu, Oct 8, 2009 at 2:15 PM, wrote:
> On 6 Oct, 02:47 pm, petshm...@googlemail.com wrote:
>>Hi,
>>
>>calling in client code self.cred.login() I can't print login result.
>>What I'm doing wrong? login method returns deffered, but should yield
>>result from callRemote method.
>>
>>Thanks for any help!
>>
>>Pet
>>
>>class Cred:
>>
>> �...@inlinecallbacks
>> def login(self):
>> proxy = Proxy(LOGIN_PROXY)
>> l = {"user":LOGIN_USERNAME, "pass":LOGIN_PASSWD}
>> loginResult = yield proxy.callRemote('login', l)
>> print "RESULT", loginResult.result
>> returnValue(loginResult)
>
> It's hard to tell what's going wrong since this example isn't complete.
> If I assume I know what Proxy does, then the code basically looks right
> - although I suspect you only want to print "loginResult", not
> "loginResult.result". If you can post an sscce - http://sscce.org/ -
> someone might be able to be of more help.
Hi,
I've misunderstood the concept of inlinecallbacks. I've thought they
makes deferred blocking "again" so execution of code stops until
result is available and didn't catch any errors of Proxy. Because
connection failed, there was no error and no result. I've finally
realized that inlinecallbacks do not change behavior of deferred and
now I simply use my login function as deferred with callbacks and
errbacks
Pet
>
> Jean-Paul
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Log rotates not as expected
Hi, in code below I try to rotate log files if they reach 500 limit, but this happens only to every 5 files, ls -althr: -rw-r--r-- 1 root root 977K 2010-01-19 19:03 /var/log/my.log.12 -rw-r--r-- 1 root root 977K 2010-01-19 19:55 /var/log/my.log.10 -rw-r--r-- 1 root root 977K 2010-01-19 20:41 /var/log/my.log.9 -rw-r--r-- 1 root root 977K 2010-01-19 21:30 /var/log/my.log.8 -rw-r--r-- 1 root root 977K 2010-01-19 22:46 /var/log/my.log.7 -rw-r--r-- 1 root root 4.8M 2010-01-20 00:19 /var/log/my.log.11 -rw-r--r-- 1 root root 977K 2010-01-20 00:19 /var/log/my.log.6 -rw-r--r-- 1 root root 977K 2010-01-20 09:17 /var/log/my.log.4 -rw-r--r-- 1 root root 977K 2010-01-20 10:45 /var/log/my.log.3 -rw-r--r-- 1 root root 977K 2010-01-20 12:04 /var/log/my.log.2 -rw-r--r-- 1 root root 977K 2010-01-20 12:59 /var/log/my.log.1 -rw-r--r-- 1 root root 4.2M 2010-01-20 13:26 /var/log/my.log.5 -rw-r--r-- 1 root root 301K 2010-01-20 13:26 /var/log/my.log it writes simultaneously to both my.log.5 and my.log. It is not big problem, but in this way I have only recent files, because they grows quickly Pet from twisted.python import log from twisted.python import logfile from twisted.application import service class MyLog(log.FileLogObserver): def emit(self, logEntryDict): log.FileLogObserver.timeFormat = '%Y-%m-%d %H:%M:%S' log.FileLogObserver.emit(self, logEntryDict) class MyLogService(service.Service): def __init__(self, logName, logDir): self.logName = logName self.logDir = logDir # near 5mb self.maxLogSize = 500 def startService(self): # logfile is a file-like object that supports rotation self.logFile = logfile.LogFile( self.logName, self.logDir, rotateLength=self.maxLogSize, maxRotatedFiles=50) #self.logFile.rotate() # force rotation each time restarted self.loclog = MyLog(self.logFile) self.loclog.start() def stopService(self): self.loclog.stop() self.logFile.close() del(self.logFile) ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Log rotates not as expected
On Thu, Jan 21, 2010 at 7:02 PM, Maarten ter Huurne wrote: > On Thursday 21 January 2010, Lucas Taylor wrote: > >> This can occur if you have multiple instances of logfile.LogFile setup >> to manage the same file. The default rotation is 1MB, so maybe you have >> another logfile.LogFile somewhere? > > Maybe twistd's log rotation? Yes, may be. I start my daemon with /usr/bin/twistd -y mydaemon.py --logfile=/var/log/my.log --pidfile=/var/lock/mydaemon.pid How do I start twistd, so it doesn't produce own log file? Thanks for help! Pet > > Bye, > Maarten > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Log rotates not as expected
On Fri, Jan 22, 2010 at 12:13 AM, Lucas Taylor wrote:
> On 1/21/10 11:17 AM, Pet wrote:
>> On Thu, Jan 21, 2010 at 7:02 PM, Maarten ter Huurne
>> wrote:
>>> On Thursday 21 January 2010, Lucas Taylor wrote:
>>>
>>>> This can occur if you have multiple instances of logfile.LogFile setup
>>>> to manage the same file. The default rotation is 1MB, so maybe you have
>>>> another logfile.LogFile somewhere?
>>> Maybe twistd's log rotation?
>>
>> Yes, may be. I start my daemon with
>> /usr/bin/twistd -y mydaemon.py --logfile=/var/log/my.log
>> --pidfile=/var/lock/mydaemon.pid
>>
>> How do I start twistd, so it doesn't produce own log file?
>>
>> Thanks for help!
>>
>> Pet
>>
>
> You can customize the application to use your logfile and observer:
> http://twistedmatrix.com/documents/current/core/howto/application.html#auto6
>
> e.g
> class MyLog(log.FileLogObserver):
> def emit(self, logEntryDict):
> log.FileLogObserver.timeFormat = '%Y-%m-%d %H:%M:%S'
> log.FileLogObserver.emit(self, logEntryDict)
>
> maxLogSize = 500
> logFile = logfile.LogFile("my.log", "/var/log", rotateLength=maxLogSize,
> maxRotatedFiles=50)
>
> application = service.Application("myapp")
> application.setComponent(log.ILogObserver, MyLog(logFile).emit)
Thanks for suggestion. I'll try it out as soon as I can. Currently I
do it in that way:
application = service.Application("MyService")
myLogService = myLogService(LOG_NAME, LOG_DIR)
myLogService.setServiceParent(application)
what is the difference between creating service and setting
setServiceParent and setComponent?
Pet
>
>
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Log rotates not as expected
On Fri, Jan 22, 2010 at 10:21 PM, Lucas Taylor wrote:
> On 1/22/10 3:15 AM, Pet wrote:
>> On Fri, Jan 22, 2010 at 12:13 AM, Lucas Taylor
>> wrote:
>>> On 1/21/10 11:17 AM, Pet wrote:
>>>> On Thu, Jan 21, 2010 at 7:02 PM, Maarten ter Huurne
>>>> wrote:
>>>>> On Thursday 21 January 2010, Lucas Taylor wrote:
>>>>>
>>>>>> This can occur if you have multiple instances of logfile.LogFile setup
>>>>>> to manage the same file. The default rotation is 1MB, so maybe you have
>>>>>> another logfile.LogFile somewhere?
>>>>> Maybe twistd's log rotation?
>>>> Yes, may be. I start my daemon with
>>>> /usr/bin/twistd -y mydaemon.py --logfile=/var/log/my.log
>>>> --pidfile=/var/lock/mydaemon.pid
>>>>
>>>> How do I start twistd, so it doesn't produce own log file?
>>>>
>>>> Thanks for help!
>>>>
>>>> Pet
>>>>
>>> You can customize the application to use your logfile and observer:
>>> http://twistedmatrix.com/documents/current/core/howto/application.html#auto6
>>>
>>> e.g
>>> class MyLog(log.FileLogObserver):
>>> def emit(self, logEntryDict):
>>> log.FileLogObserver.timeFormat = '%Y-%m-%d %H:%M:%S'
>>> log.FileLogObserver.emit(self, logEntryDict)
>>>
>>> maxLogSize = 500
>>> logFile = logfile.LogFile("my.log", "/var/log", rotateLength=maxLogSize,
>>> maxRotatedFiles=50)
>>>
>>> application = service.Application("myapp")
>>> application.setComponent(log.ILogObserver, MyLog(logFile).emit)
>>
>> Thanks for suggestion. I'll try it out as soon as I can. Currently I
>> do it in that way:
>>
>> application = service.Application("MyService")
>> myLogService = myLogService(LOG_NAME, LOG_DIR)
>> myLogService.setServiceParent(application)
>>
>>
>> what is the difference between creating service and setting
>> setServiceParent and setComponent?
>>
>> Pet
>>
> http://twistedmatrix.com/documents/current/core/howto/application.html
>
> The important aspect is the setComponent api. This is the part that lets
> you override the default logging behavior of twistd. It has nothing to
> do with the services that you register with the application using
> setServiceParent.
>
> If your MyLogService only does what you originally posted, you probably
> don't need all of that machinery. Using twistd will take care of
> starting and stopping logging for you.
>
> But, if you really want to use your service (say you want to force
> rotation on a restart), then you can do so. You just need to set the
> ILogObserver component on the application using your observer's emit
> function.
>
> e.g
>
> application = service.Application("MyService")
> myLogService = myLogService(LOG_NAME, LOG_DIR)
> myLogService.setServiceParent(application)
> application.setComponent(log.ILogObserver, myLogService.loclog.emit)
>
> Note that this won't work with your original MyLogService implementation
> without some reorganization (move logfile and loclog creation up to
> __init__)
>
Hi,
I'm getting an error while starting my service:
Failed to load application: 'module' object has no attribute 'ILogObserver'
I've imported log with
from twisted.python import log
Have no idea what is wrong...
Pet
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Log rotates not as expected
On Thu, Jan 28, 2010 at 9:46 AM, Pet wrote:
> On Fri, Jan 22, 2010 at 10:21 PM, Lucas Taylor
> wrote:
>> On 1/22/10 3:15 AM, Pet wrote:
>>> On Fri, Jan 22, 2010 at 12:13 AM, Lucas Taylor
>>> wrote:
>>>> On 1/21/10 11:17 AM, Pet wrote:
>>>>> On Thu, Jan 21, 2010 at 7:02 PM, Maarten ter Huurne
>>>>> wrote:
>>>>>> On Thursday 21 January 2010, Lucas Taylor wrote:
>>>>>>
>>>>>>> This can occur if you have multiple instances of logfile.LogFile setup
>>>>>>> to manage the same file. The default rotation is 1MB, so maybe you have
>>>>>>> another logfile.LogFile somewhere?
>>>>>> Maybe twistd's log rotation?
>>>>> Yes, may be. I start my daemon with
>>>>> /usr/bin/twistd -y mydaemon.py --logfile=/var/log/my.log
>>>>> --pidfile=/var/lock/mydaemon.pid
>>>>>
>>>>> How do I start twistd, so it doesn't produce own log file?
>>>>>
>>>>> Thanks for help!
>>>>>
>>>>> Pet
>>>>>
>>>> You can customize the application to use your logfile and observer:
>>>> http://twistedmatrix.com/documents/current/core/howto/application.html#auto6
>>>>
>>>> e.g
>>>> class MyLog(log.FileLogObserver):
>>>> def emit(self, logEntryDict):
>>>> log.FileLogObserver.timeFormat = '%Y-%m-%d %H:%M:%S'
>>>> log.FileLogObserver.emit(self, logEntryDict)
>>>>
>>>> maxLogSize = 500
>>>> logFile = logfile.LogFile("my.log", "/var/log", rotateLength=maxLogSize,
>>>> maxRotatedFiles=50)
>>>>
>>>> application = service.Application("myapp")
>>>> application.setComponent(log.ILogObserver, MyLog(logFile).emit)
>>>
>>> Thanks for suggestion. I'll try it out as soon as I can. Currently I
>>> do it in that way:
>>>
>>> application = service.Application("MyService")
>>> myLogService = myLogService(LOG_NAME, LOG_DIR)
>>> myLogService.setServiceParent(application)
>>>
>>>
>>> what is the difference between creating service and setting
>>> setServiceParent and setComponent?
>>>
>>> Pet
>>>
>> http://twistedmatrix.com/documents/current/core/howto/application.html
>>
>> The important aspect is the setComponent api. This is the part that lets
>> you override the default logging behavior of twistd. It has nothing to
>> do with the services that you register with the application using
>> setServiceParent.
>>
>> If your MyLogService only does what you originally posted, you probably
>> don't need all of that machinery. Using twistd will take care of
>> starting and stopping logging for you.
>>
>> But, if you really want to use your service (say you want to force
>> rotation on a restart), then you can do so. You just need to set the
>> ILogObserver component on the application using your observer's emit
>> function.
>>
>> e.g
>>
>> application = service.Application("MyService")
>> myLogService = myLogService(LOG_NAME, LOG_DIR)
>> myLogService.setServiceParent(application)
>> application.setComponent(log.ILogObserver, myLogService.loclog.emit)
>>
>> Note that this won't work with your original MyLogService implementation
>> without some reorganization (move logfile and loclog creation up to
>> __init__)
>>
>
> Hi,
>
> I'm getting an error while starting my service:
>
> Failed to load application: 'module' object has no attribute 'ILogObserver'
>
> I've imported log with
>
> from twisted.python import log
>
>
> Have no idea what is wrong...
Ok, there is no ILogObserver in log module.
I've Twisted 8.1.0 installed.
>
>
> Pet
>
>
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] multiple versions of twisted on same machine
Hi, I'd like to upgrade twisted to the newest version and obviously first test if my application works fine with new version. What is the best (safest) way to do it? Thanks for help, Pet ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] multiple versions of twisted on same machine
On Thu, Feb 11, 2010 at 1:55 PM, Matt Bone wrote: > You could try virtualenv: > > http://pypi.python.org/pypi/virtualenv > > And have one virtual environment with Twisted 9.0 and another with > Twisted 8.2, etc, etc. > Thanks! Will try it out > --matt > > On Thu, Feb 11, 2010 at 6:41 AM, Pet wrote: >> Hi, >> >> I'd like to upgrade twisted to the newest version and obviously first >> test if my application works fine with new version. What is the best >> (safest) way to do it? >> >> Thanks for help, Pet >> >> ___ >> Twisted-Python mailing list >> Twisted-Python@twistedmatrix.com >> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python >> > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] profiling code that runs in threads
Hello! I've tried to profile my twisted server with --profile=path --profiler=cProfile --savestats options for twistd and it show results only for functions which are in main thread and it is not really helpful, because "dirty work" is done in threads. Is it possible to profile functions which runs in other threads? Thanks! Pet ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] multiple versions of twisted on same machine
On Sat, Feb 13, 2010 at 6:05 AM, Glyph Lefkowitz wrote: > On Feb 11, 2010, at 5:27 PM, Timothy Allen wrote: > >> On Thu, 11 Feb 2010 11:54:47 -0600 >> Kevin Horn wrote: >>> Not entirely related to this discussion (though not entirely >>> unrelated), I think that Twisted's whole release/distribution system >>> needs to be revamped a little. I've been meaning to talk to radix >>> about this, but haven't found the time...maybe this will give me the >>> kick in the pants I needed... >> >> Oh ho ho ho ho, boy. >> >> As one who has tinkered with Twisted's release/distribution system >> (unsuccessfully so far - see ticket 1696 and 4138), I'd be very >> interested to hear what you propose. > > On the contrary - I think your progress on 1696 especially has been *very* > successful! perhaps not quite *complete* yet, though :). > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > Hi, I can't install Twisted in VirtualEnv. Can someone explain what is wrong, please? Thanks! (newtwisted)hostname:/usr/local/development/newtwisted/bin# easy_install Twisted Searching for Twisted Reading http://pypi.python.org/simple/Twisted/ Reading http://www.twistedmatrix.com Reading http://twistedmatrix.com/products/download Reading http://twistedmatrix.com/projects/core/ Reading http://twistedmatrix.com/ Reading http://tmrc.mit.edu/mirror/twisted/Twisted/8.2/ Reading http://tmrc.mit.edu/mirror/twisted/Twisted/8.1/ Reading http://tmrc.mit.edu/mirror/twisted/Twisted/10.0/ Reading http://tmrc.mit.edu/mirror/twisted/Twisted/9.0/ Best match: Twisted 10.0.0 Downloading http://tmrc.mit.edu/mirror/twisted/Twisted/10.0/Twisted-10.0.0.tar.bz2 Processing Twisted-10.0.0.tar.bz2 Running Twisted-10.0.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-9n_UrX/Twisted-10.0.0/egg-dist-tmp-dt-vez twisted/runner/portmap.c:10:20: error: Python.h: No such file or directory twisted/runner/portmap.c:14: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token twisted/runner/portmap.c:31: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token twisted/runner/portmap.c:45: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘PortmapMethods’ twisted/runner/portmap.c: In function ‘initportmap’: twisted/runner/portmap.c:55: warning: implicit declaration of function ‘Py_InitModule’ twisted/runner/portmap.c:55: error: ‘PortmapMethods’ undeclared (first use in this function) twisted/runner/portmap.c:55: error: (Each undeclared identifier is reported only once twisted/runner/portmap.c:55: error: for each function it appears in.) error: Setup script exited with error: command 'gcc' failed with exit status 1 ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] multiple versions of twisted on same machine
On Tue, Apr 27, 2010 at 5:28 PM, wrote: > On 02:43 pm, petshm...@googlemail.com wrote: >> >>Hi, >> >>I can't install Twisted in VirtualEnv. Can someone explain what is >>wrong, please? Thanks! >> >> >>(newtwisted)hostname:/usr/local/development/newtwisted/bin# >>easy_install Twisted >>[snip] >>Running Twisted-10.0.0/setup.py -q bdist_egg --dist-dir >>/tmp/easy_install-9n_UrX/Twisted-10.0.0/egg-dist-tmp-dt-vez >>twisted/runner/portmap.c:10:20: error: Python.h: No such file or >>directory > ^^ > > You need to install the Python development headers. This is usually in > a package named "python-dev" or "python-devel" or something like that. thanks, it worked! > > Jean-Paul > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] print unicode
Hello!
I'm using Twisted 10.0 and as usually sometime print debug infos with
myunicodestr.encode('UTF-8') which are saved to logfile, but since
using twisted 10 I'm getting
UnicodeEncodeError: 'ascii' codec can't encode characters...
type(myunicodestr) returns
What is the problem here?
Thanks!
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] print unicode
On Wed, May 5, 2010 at 2:05 PM, Maarten ter Huurne
wrote:
> On Wednesday 05 May 2010, Pet wrote:
>
>> I'm using Twisted 10.0 and as usually sometime print debug infos with
>> myunicodestr.encode('UTF-8') which are saved to logfile, but since
>> using twisted 10 I'm getting
>>
>> UnicodeEncodeError: 'ascii' codec can't encode characters...
>
> UTF-8 uses the full 8 bits of a byte, while ASCII only uses 7, so writing
> Unicode encoded as UTF-8 to an ASCII stream is not valid.
>
> I think recent Python versions are more strict about what is written to
> stdout/stderr than older versions, it might not be related to Twisted
> itself. You can specify a different encoding for stdin/out/err by setting
> the PYTHONIOENCODING environment variable.
Hi Maarten!
Thanks for help!
Unfortunately, my Python installation is 2.5.2 and PYTHONIOENCODING is
introduced in 2.6
Pet
>
> Bye,
> Maarten
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] print unicode
On Wed, May 5, 2010 at 2:47 PM, Phil Mayers wrote:
> On 05/05/10 13:31, Pet wrote:
>> On Wed, May 5, 2010 at 2:05 PM, Maarten ter Huurne
>> wrote:
>>> On Wednesday 05 May 2010, Pet wrote:
>>>
>>>> I'm using Twisted 10.0 and as usually sometime print debug infos with
>>>> myunicodestr.encode('UTF-8') which are saved to logfile, but since
>>>> using twisted 10 I'm getting
>>>>
>>>> UnicodeEncodeError: 'ascii' codec can't encode characters...
>>>
>>> UTF-8 uses the full 8 bits of a byte, while ASCII only uses 7, so writing
>>> Unicode encoded as UTF-8 to an ASCII stream is not valid.
>>>
>>> I think recent Python versions are more strict about what is written to
>>> stdout/stderr than older versions, it might not be related to Twisted
>>> itself. You can specify a different encoding for stdin/out/err by setting
>>> the PYTHONIOENCODING environment variable.
>>
>> Hi Maarten!
>>
>> Thanks for help!
>> Unfortunately, my Python installation is 2.5.2 and PYTHONIOENCODING is
>> introduced in 2.6
>
> I think this is highly dependent on your OS environment. For example:
>
> Python 2.4.3 (#1, Oct 23 2006, 14:19:47)
> [GCC 4.1.1 20060525 (Red Hat 4.1.1-1)] o
> Type "help", "copyright", "credits" or "
> >>> import sys
> >>> sys.getdefaultencoding()
> 'ascii'
> >>> print unichr(163)
> £
on python console it works for me, but not in application, if the
string to be printed is fetched from db which is utf-8
>
>
> [p...@wildfire ~]$ echo $LANG
> en_GB.UTF-8
>
> ...but:
>
> LANG=C python
> Python 2.4.3 (#1, Oct 23 2006, 14:19:47)
> [GCC 4.1.1 20060525 (Red Hat 4.1.1-1)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> print unichr(163)
> Traceback (most recent call last):
> File "", line 1, in ?
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xa3' in
> position 0: ordinal not in range(128)
>
> ...i.e. here I can just print unicode characters, with nothing
> particularly special, provided my environment variables are set right.
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] print unicode
On Wed, May 5, 2010 at 2:49 PM, Itamar Turner-Trauring
wrote:
> On Wed, 2010-05-05 at 13:45 +0200, Pet wrote:
>> Hello!
>>
>> I'm using Twisted 10.0 and as usually sometime print debug infos with
>> myunicodestr.encode('UTF-8') which are saved to logfile, but since
>> using twisted 10 I'm getting
>>
>> UnicodeEncodeError: 'ascii' codec can't encode characters...
>>
>> type(myunicodestr) returns
>>
>>
>> What is the problem here?
>
> This works fine for me (Twisted trunk):
>
> $ python2.5 -c "import sys; from twisted.python import log; \
> log.startLogging(file('/tmp/log', 'w')); print \
> u'\u1234'.encode('UTF-8')"
> $ cat /tmp/log
> 2010-05-05 08:48:40-0400 [-] Log opened.
> 2010-05-05 08:48:40-0400 [-] ሴ
>
> Can you include a minimal reproducing example?
If I print as you do it works, but my string is fetched from database
and only then it fails
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] print unicode
On Wed, May 5, 2010 at 4:15 PM, Pet wrote:
> On Wed, May 5, 2010 at 2:49 PM, Itamar Turner-Trauring
> wrote:
>> On Wed, 2010-05-05 at 13:45 +0200, Pet wrote:
>>> Hello!
>>>
>>> I'm using Twisted 10.0 and as usually sometime print debug infos with
>>> myunicodestr.encode('UTF-8') which are saved to logfile, but since
>>> using twisted 10 I'm getting
>>>
>>> UnicodeEncodeError: 'ascii' codec can't encode characters...
>>>
>>> type(myunicodestr) returns
>>>
>>>
>>> What is the problem here?
>>
>> This works fine for me (Twisted trunk):
>>
>> $ python2.5 -c "import sys; from twisted.python import log; \
>> log.startLogging(file('/tmp/log', 'w')); print \
>> u'\u1234'.encode('UTF-8')"
>> $ cat /tmp/log
>> 2010-05-05 08:48:40-0400 [-] Log opened.
>> 2010-05-05 08:48:40-0400 [-] ሴ
>>
>> Can you include a minimal reproducing example?
>
> If I print as you do it works, but my string is fetched from database
> and only then it fails
It's pretty weird. I've send as parameter {'s': u'c\u0142a'} to
twisted xml-rpc server after it was restarted and it has printed
param['s'].encode('UTF-8') without errors. Immidiately after that I've
send the same request again and it failed to print it. I've restarted
the server again and at the first request it prints without errors,
all other requests raise exceptions. So it has nothing to do with
database.
>
>>
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] print unicode
On Wed, May 5, 2010 at 4:29 PM, Pet wrote:
> On Wed, May 5, 2010 at 4:15 PM, Pet wrote:
>> On Wed, May 5, 2010 at 2:49 PM, Itamar Turner-Trauring
>> wrote:
>>> On Wed, 2010-05-05 at 13:45 +0200, Pet wrote:
>>>> Hello!
>>>>
>>>> I'm using Twisted 10.0 and as usually sometime print debug infos with
>>>> myunicodestr.encode('UTF-8') which are saved to logfile, but since
>>>> using twisted 10 I'm getting
>>>>
>>>> UnicodeEncodeError: 'ascii' codec can't encode characters...
>>>>
>>>> type(myunicodestr) returns
>>>>
>>>>
>>>> What is the problem here?
>>>
>>> This works fine for me (Twisted trunk):
>>>
>>> $ python2.5 -c "import sys; from twisted.python import log; \
>>> log.startLogging(file('/tmp/log', 'w')); print \
>>> u'\u1234'.encode('UTF-8')"
>>> $ cat /tmp/log
>>> 2010-05-05 08:48:40-0400 [-] Log opened.
>>> 2010-05-05 08:48:40-0400 [-] ሴ
>>>
>>> Can you include a minimal reproducing example?
>>
>> If I print as you do it works, but my string is fetched from database
>> and only then it fails
>
> It's pretty weird. I've send as parameter {'s': u'c\u0142a'} to
> twisted xml-rpc server after it was restarted and it has printed
> param['s'].encode('UTF-8') without errors. Immidiately after that I've
> send the same request again and it failed to print it. I've restarted
> the server again and at the first request it prints without errors,
> all other requests raise exceptions. So it has nothing to do with
> database.
Now, I'm getting Exception with
File
"/usr/local/tw10/lib/python2.5/site-packages/Twisted-10.0.0-py2.5-linux-x86_64.egg/twisted/python/log.py",
line 555, in write
d = (self.buf + data).split('\n')
exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte
0xc5 in position 4: ordinal not in range(128)
>
>>
>>>
>>>
>>> ___
>>> Twisted-Python mailing list
>>> Twisted-Python@twistedmatrix.com
>>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>>>
>>
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] print unicode
On Wed, May 5, 2010 at 4:54 PM, Itamar Turner-Trauring
wrote:
> On Wed, 2010-05-05 at 16:47 +0200, Pet wrote:
>> Now, I'm getting Exception with
>>
>> File
>> "/usr/local/tw10/lib/python2.5/site-packages/Twisted-10.0.0-py2.5-linux-x86_64.egg/twisted/python/log.py",
>> line 555, in write
>> d = (self.buf + data).split('\n')
>> exceptions.UnicodeDecodeError: 'ascii' codec can't decode byte
>> 0xc5 in position 4: ordinal not in range(128)
>
> Are you logging/printing unencoded unicode strings (i.e. type(s) ==
> unicode)? Twisted does not support that.
No, this exception occurs if I do print myunicodestring.encode('UTF-8')
As I said before, it doesn't happen at first request after server restart.
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] multiline log entries
Hello! Is it possible to print multiline debug messages into log? If print a query it looks like this 2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15] SELECT * 2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15] FROM foo AS a 2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15] JOIN bar AS b ON a.id = b.id 2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15] WHERE a.id = 1 2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15] ORDER BY id DESC 2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15] LIMIT 1; it would be more practical to have only the query without timestamp on other things Thanks! ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] multiline log entries
On Thu, May 6, 2010 at 4:37 PM, wrote:
> On 02:30 pm, petshm...@googlemail.com wrote:
>>Hello!
>>
>>Is it possible to print multiline debug messages into log?
>>
>>If print a query it looks like this
>>
>>
>>2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15]
>>SELECT *
>>2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15]
>>FROM foo AS a
>>2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15]
>>JOIN bar AS b ON a.id = b.id
>>2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15]
>>WHERE a.id = 1
>>2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15]
>>ORDER BY id DESC
>>2010-05-06 10:10:07 [HTTPChannel,0,192.168.2.15]
>>LIMIT 1;
>>
>>
>>it would be more practical to have only the query without timestamp on
>>other things
>
> Use log.msg, instead. `print` integration is meant as a debugging aid,
> not a general logging facility.
Thanks! I've thought they're equivalent
>
> >>> from twisted.python.log import startLogging
> >>> from sys import stdout
> >>> startLogging(stdout)
> 2010-05-06 10:36:41-0400 [-] Log opened.
> 2010-05-06 10:36:41-0400 [-] instance at 0xb766424c>
> >>> from twisted.python.log import msg
> >>> print 'hello\nworld'
> 2010-05-06 10:36:50-0400 [-] hello
> 2010-05-06 10:36:50-0400 [-] world
> >>> msg('hello\nworld')
> 2010-05-06 10:36:54-0400 [-] hello
> world
> >>>
> Jean-Paul
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] getting all results from DeferredList
Hello,
I'm trying to process results from several Deferreds (d1, d2) with
DeferredList, merging it into one list and sending back to client.
While merging works, the client has no results. What is the way to do
tasks (db queries in my case) in parallel and then merge results?
BTW, are nested runInteraction ok?
Thanks for help!
Pet
class Test:
def __init__(self, cursor):
self.cursor = cursor
def test(db, params):
if params.get('query'):
params['q'] = self.someFunc3(db, params)
def processResults(results, out):
#merge results into one
for _, r in results:
out.extend(r)
return out
out = []
d1 = self.cursor.runInteraction(self.someFunc, params)
d2 = self.cursor.runInteraction(self.someFunc2, params)
d = DeferredList([d1,d2])
d.addCallback(processResults, out)
d.addErrback(log.err)
return d
class XMLRPCProtokoll(xmlrpc.XMLRPC):
def __init__(self):
self.db = adbapi.ConnectionPool()
xmlrpc.XMLRPC.__init__(self, True)
def xmlrpc_test(self, param):
t = Test(self.db)
return self.db.runInteraction(t.test, param)
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] getting all results from DeferredList
On Wed, Jun 30, 2010 at 8:42 PM, wrote:
> On 02:41 pm, petshm...@googlemail.com wrote:
>>Hello,
>>
>>I'm trying to process results from several Deferreds (d1, d2) with
>>DeferredList, merging it into one list and sending back to client.
>
> You might like twisted.internet.defer.gatherResults.
>>
>>While merging works, the client has no results. What is the way to do
>>tasks (db queries in my case) in parallel and then merge results?
I mean, running functions which performs db queries asynchronously and
then merge results
>
> I don't think I understand this part of the question.
>>BTW, are nested runInteraction ok?
>
> Not really. Twisted APIs are almost all required to be invoked in the
> reactor thread. Nested runInteraction calls would mean calling
> runInteraction in some other thread, and that's not allowed.
That is why it not worked. I should redesign my application.
Thanks!
>
> Jean-Paul
>>Thanks for help!
>>Pet
>>
>>
>>class Test:
>> def __init__(self, cursor):
>> self.cursor = cursor
>>
>> def test(db, params):
>> if params.get('query'):
>> params['q'] = self.someFunc3(db, params)
>>
>> def processResults(results, out):
>> #merge results into one
>> for _, r in results:
>> out.extend(r)
>> return out
>>
>> out = []
>> d1 = self.cursor.runInteraction(self.someFunc, params)
>> d2 = self.cursor.runInteraction(self.someFunc2, params)
>> d = DeferredList([d1,d2])
>> d.addCallback(processResults, out)
>> d.addErrback(log.err)
>> return d
>>
>>class XMLRPCProtokoll(xmlrpc.XMLRPC):
>> def __init__(self):
>> self.db = adbapi.ConnectionPool()
>> xmlrpc.XMLRPC.__init__(self, True)
>>
>> def xmlrpc_test(self, param):
>> t = Test(self.db)
>> return self.db.runInteraction(t.test, param)
>>
>>___
>>Twisted-Python mailing list
>>Twisted-Python@twistedmatrix.com
>>http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] xmlrpc protocoll instanze per connection?
Hello, I've started learning twisted by implementing linereceiver, where for each new connection a new instance of MyProtocol is created: class MyFactory(protocol.ServerFactory): protocol = MyProtocol def __init__(self): self.db = adbapi.ConnectionPool() class MyService(internet.TCPServer): def __init__(self): print "Starting Service..." internet.TCPServer.__init__(self, PORT, MyFactory()) def main(): reactor.listenTCP(settings.PORT, MyFactory()) reactor.run() if __name__ == '__main__': main() Now, for XMLRPC this looks for me like Protocol instance is reused for all connection. If it is so, can I change this behavior, so I have one instance of Protocol per connection? Thanks for help! Pet class MyXMLRPCFactory(server.Site): def __init__(self): self.db = adbapi.ConnectionPool() self.r = MyXMLRPCProtokoll(self) server.Site.__init__(self,self.r) class MyXMLRPCService(internet.TCPServer): def __init__(self): internet.TCPServer.__init__(self, settings.XMLRPC_PORT, MyXMLRPCFactory()) def main(): from twisted.internet import reactor r = MyXMLRPCProtokoll() reactor.listenTCP(settings.XMLRPC_PORT, server.Site(r)) reactor.run() if __name__ == '__main__': main() ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] xmlrpc protocoll instanze per connection?
On Wed, Aug 4, 2010 at 12:25 PM, Alan Franzoni wrote: > Just ovverride the "protocol" attribute on your factory with the > callable you want to be called when a new protocol needs to be Hello Alan, AFAIU this works only if Factory is subclassing protocol.ServerFactory, XMLRPCFactory subclasses server.Site, where protocol attribute is not present Thanks for suggestions! > instanced. I'd suppose you'd probably want to subclass HTTPChannel or > something like that. > > > > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] xmlrpc protocoll instanze per connection?
On Wed, Aug 4, 2010 at 2:13 PM, Itamar Turner-Trauring wrote: > On Wed, 2010-08-04 at 12:00 +0200, Pet wrote: > >> Now, for XMLRPC this looks for me like Protocol instance is reused for >> all connection. If it is so, can I change this behavior, so I have one >> instance of Protocol per connection? > > XMLRPC is a Resource, not a Protocol: Resources handle HTTP *requests*. > The reason Resources don't care about connections is that HTTP doesn't > care about connections: you can send multiple requests over the same > connection, or one request per connection, and the semantics are the > same either way. That make sense, thanks. Is there a way to isolate requests from each other? Pet > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] xmlrpc protocoll instanze per connection?
On Thu, Aug 5, 2010 at 3:32 PM, Alan Franzoni wrote: > On 8/4/10 1:30 PM, Pet wrote: > >> AFAIU this works only if Factory is subclassing >> protocol.ServerFactory, XMLRPCFactory subclasses server.Site, where >> protocol attribute is not present > > server.Site subclasses http.HTTPFactory which, in turns, subclasses > protocol.ServerFactory: > > http://twistedmatrix.com/documents/10.0.0/api/twisted.web.server.Site.html > > > by the way, if I now understand your question: just subclass XMLRPC and > create your own xmlrpc resource class by adding methods like > > xmlrpc_somemethod > xmlrpc_othermethod > > then add such handler to your server.Site, and you're done. Your method > will be invoked once per request. what I've tried to do is to have objects, which were members of XMLRPC. And these objects would have some internal state which is isolated per request. But each new request has changed this objects, because xmlrpc is created once and not for each request or connection. I've solved this by modifying render_POST where I create those objects before calling requested function and so they lives only there without being overwritten by new request. Ugh... It's might be ugly, but works for me Pet > > Alan Franzoni > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] xmlrpc protocoll instanze per connection?
On Thu, Aug 5, 2010 at 5:01 PM, wrote: > On 02:21 pm, petshm...@googlemail.com wrote: >> >> On Thu, Aug 5, 2010 at 3:32 PM, Alan Franzoni wrote: >>> >>> On 8/4/10 1:30 PM, Pet wrote: >>>> >>>> AFAIU this works only if Factory is subclassing >>>> protocol.ServerFactory, XMLRPCFactory subclasses server.Site, where >>>> protocol attribute is not present >>> >>> server.Site subclasses http.HTTPFactory which, in turns, subclasses >>> protocol.ServerFactory: >>> >>> >>> http://twistedmatrix.com/documents/10.0.0/api/twisted.web.server.Site.html >>> >>> >>> by the way, if I now understand your question: just subclass XMLRPC and >>> create your own xmlrpc resource class by adding methods like >>> >>> xmlrpc_somemethod >>> xmlrpc_othermethod >>> >>> then add such handler to your server.Site, and you're done. Your method >>> will be invoked once per request. >> >> what I've tried to do is to have objects, which were members of >> XMLRPC. And these objects would have some internal state which is >> isolated per request. But each new request has changed this objects, >> because xmlrpc is created once and not for each request or connection. >> I've solved this by modifying render_POST where I create those objects >> before calling requested function and so they lives only there without >> being overwritten by new request. Ugh... It's might be ugly, but works >> for me > > If you want a new XMLRPC instance per request, then create a new one per That is what I've wanted > request. http://twistedmatrix.com/documents/current/web/howto/web-in-60 > /dynamic-dispatch.html may help. Thanks, will check it out > > Re-using a single XMLRPC instance and mangling its state in render_POST will > break as soon as your server receives two overlapping requests. I'm making only local variables in render_POST, not class wide, seems to work even with 2 overlapping requests. Pet > > Jean-Paul > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] xmlrpc protocoll instanze per connection?
On Thu, Aug 5, 2010 at 5:01 PM, wrote: > On 02:21 pm, petshm...@googlemail.com wrote: >> >> On Thu, Aug 5, 2010 at 3:32 PM, Alan Franzoni wrote: >>> >>> On 8/4/10 1:30 PM, Pet wrote: >>>> >>>> AFAIU this works only if Factory is subclassing >>>> protocol.ServerFactory, XMLRPCFactory subclasses server.Site, where >>>> protocol attribute is not present >>> >>> server.Site subclasses http.HTTPFactory which, in turns, subclasses >>> protocol.ServerFactory: >>> >>> >>> http://twistedmatrix.com/documents/10.0.0/api/twisted.web.server.Site.html >>> >>> >>> by the way, if I now understand your question: just subclass XMLRPC and >>> create your own xmlrpc resource class by adding methods like >>> >>> xmlrpc_somemethod >>> xmlrpc_othermethod >>> >>> then add such handler to your server.Site, and you're done. Your method >>> will be invoked once per request. >> >> what I've tried to do is to have objects, which were members of >> XMLRPC. And these objects would have some internal state which is >> isolated per request. But each new request has changed this objects, >> because xmlrpc is created once and not for each request or connection. >> I've solved this by modifying render_POST where I create those objects >> before calling requested function and so they lives only there without >> being overwritten by new request. Ugh... It's might be ugly, but works >> for me > > If you want a new XMLRPC instance per request, then create a new one per > request. http://twistedmatrix.com/documents/current/web/howto/web-in-60 > /dynamic-dispatch.html may help. This works for me too. getChild() returns for each request new instance. When is this object destroyed? I've putted print into __del__, but can't see it in log. Pet > > Re-using a single XMLRPC instance and mangling its state in render_POST will > break as soon as your server receives two overlapping requests. > > Jean-Paul > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] xmlrpc protocoll instanze per connection?
On Mon, Aug 9, 2010 at 3:18 PM, wrote: > On 12:07 pm, petshm...@googlemail.com wrote: >> >> [snip] >>> >>> If you want a new XMLRPC instance per request, then create a new one per >>> request. http://twistedmatrix.com/documents/current/web/howto/web- in-60 >>> /dynamic-dispatch.html may help. >> >> This works for me too. getChild() returns for each request new >> instance. When is this object destroyed? I've putted print into >> __del__, but can't see it in log. > > At the same time any Python object is destroyed, when there are no more > references to it. Twisted won't keep any references to it after the > response has been completely generated. So as long as your code also keeps > no references, it will be destroyed shortly after each request. getChild() is only defined in my code, I'm not calling it anywhere. so it must be framework holding reference returned by getChild. Well now after some minutes they were destroyed. Thanks Pet > > Jean-Paul > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] timeout in agent
Hello! I have modified Agent.request method (twisted.web.client.Agent) to pass timeout parameter to d = cc.connectTCP(host, port, timeout=timeout) but it looks like timeout is ignored (cbConnected is called). Is this known issue? Thanks! ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] timeout in agent
On Mon, Aug 30, 2010 at 1:56 PM, Itamar Turner-Trauring wrote: > On Mon, 2010-08-30 at 10:54 +0200, Pet wrote: >> Hello! >> >> I have modified Agent.request method (twisted.web.client.Agent) to >> pass timeout parameter to >> d = cc.connectTCP(host, port, timeout=timeout) >> >> but it looks like timeout is ignored (cbConnected is called). Is this >> known issue? > > Are you sure the connection happened after the timeout, not before? > Also, keep in mind that the timeout is only on the connection *attempt*, > once you're connected the timeout no longer applies. You are right, connection was maid. But the data wasn't there. Does Twisted wait for data until remote server closes connection? Thanks! > > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] How to get original error message in failure object
Hello, if an error occurs in HTTP11ClientProtocol request method (twisted.web._newclient) then error is stored in RequestGenerationFailed Exception and wrapped in Failure object: def ebRequestWriting(err): if self._state == 'TRANSMITTING': self._state = 'GENERATION_FAILED' self.transport.loseConnection() self._finishedRequest.errback( Failure(RequestGenerationFailed([err]))) else: log.err(err, "foo") but if I print detailed traceback of failure in my code it doesn't show all information it contains Original Failure (err in function above): [Failure instance: Traceback: : Data must not be unicode This is detailed traceback, where I don't see the message "Data must not be unicode" *--- Failure #7 (pickled) --- Failure: twisted.web._newclient.RequestGenerationFailed: [>] *--- End of Failure #7 --- None Is there a way to get original error message and where it was raised in Twisted code? Thanks a lot! ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] How to get original error message in failure object
On Thu, Sep 9, 2010 at 2:33 PM, wrote: > On 09:42 am, petshm...@googlemail.com wrote: >>Hello, >> >>if an error occurs in HTTP11ClientProtocol request method >>(twisted.web._newclient) then error is stored in >>RequestGenerationFailed Exception and wrapped in Failure object: >> >> >> def ebRequestWriting(err): >> if self._state == 'TRANSMITTING': >> self._state = 'GENERATION_FAILED' >> self.transport.loseConnection() >> self._finishedRequest.errback( >> Failure(RequestGenerationFailed([err]))) >> else: >> log.err(err, "foo") >> >>but if I print detailed traceback of failure in my code it doesn't >>show all information it contains >> >>Original Failure (err in function above): >>[Failure instance: Traceback: : Data must >>not be unicode >> >>This is detailed traceback, where I don't see the message "Data must >>not be unicode" >> >>*--- Failure #7 (pickled) --- >>Failure: twisted.web._newclient.RequestGenerationFailed: >>[>] >>*--- End of Failure #7 --- >>None >> >> >>Is there a way to get original error message and where it was raised >>in Twisted code? > > See the (documented) `reasons` attribute of `RequestGenerationFailed`. > Thanks for hint! RequestGenerationFailed save errors in `reasons` attribute, but documentation doesn't tell how to print reasons. I'm getting just failure object in my errback. > Jean-Paul > > ___ > Twisted-Python mailing list > Twisted-Python@twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
