Re: [Twisted-Python] Instrumenting Reactors

2011-04-06 Thread Michael Thompson
On 6 April 2011 10:55, Paul Thomas wrote: > Should I just hack into the reactor somewhere? Or is there something sitting > in a library I haven't seen that will help with this? You can time blocking calls by instrumenting twisted.python.log.callWithContext and you could try writing the timing in

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread David
On 04/07/2011 03:34 PM, Tim Allen wrote: > > Who'd have guessed it'd be so complicated to associate keys with values? If that's the only thing you need, .ini would work fine. Another solution would be python files with only literals, parsed through the ast module for safety. cheers, David __

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Tim Allen
On Thu, Apr 07, 2011 at 03:24:57PM +0900, David wrote: > On 04/07/2011 02:08 PM, Tim Allen wrote: > > If you need a non-Turing-complete config language and rule out .ini and > > XML, I'm not sure what's left. JSON, perhaps. > > Having had experience with JSON for configuration: it is a terrible >

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread David
On 04/07/2011 02:08 PM, Tim Allen wrote: > > If you need a non-Turing-complete config language and rule out .ini and > XML, I'm not sure what's left. JSON, perhaps. Having had experience with JSON for configuration: it is a terrible format for configuration, if only because it does not support c

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Glyph Lefkowitz
On Apr 6, 2011, at 8:35 PM, Stephen Thorne wrote: > Part of the discussion was about how to rewrite this in such a way that > no python code needs to be run in order to discover all the > tapname+description combinations that are available to twistd, this is > because of a perceived performance a

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Glyph Lefkowitz
On Apr 7, 2011, at 1:55 AM, Stephen Thorne wrote: > On 2011-04-07, Glyph Lefkowitz wrote: >> >> On Apr 7, 2011, at 1:08 AM, Tim Allen wrote: >> >>> Well, the nice thing about ConfigParser is that it's in the stdlib, and >>> people already know how to create them, and rolling >>> yet-another-con

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Stephen Thorne
On 2011-04-07, Glyph Lefkowitz wrote: > > On Apr 7, 2011, at 1:08 AM, Tim Allen wrote: > > > Well, the nice thing about ConfigParser is that it's in the stdlib, and > > people already know how to create them, and rolling > > yet-another-config-file-format seems crazy in this day and age. > > My

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Stephen Thorne
On 2011-04-07, Glyph Lefkowitz wrote: > > On Apr 7, 2011, at 1:38 AM, Stephen Thorne wrote: > > > On 2011-04-07, Glyph Lefkowitz wrote: > >> Because, frankly, Python installation tools REALLY REALLY SHOULD be > >> able to install Python files into Python packages. I'm not sure I can > >> make an

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Tristan Seligmann
On 7 Apr 2011 06:56, "Glyph Lefkowitz" wrote: >>- Each section blah blah terrible user interface stuff about 'sections' and other misfeatures of ini files. > > > I don't want a solution that is hard-coded to deal with the metadata that 'twistd' specifically needs, as Twisted plugins are alread

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Glyph Lefkowitz
On Apr 7, 2011, at 1:08 AM, Tim Allen wrote: > Well, the nice thing about ConfigParser is that it's in the stdlib, and > people already know how to create them, and rolling > yet-another-config-file-format seems crazy in this day and age. My point was really that people think they know how to cr

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Glyph Lefkowitz
On Apr 7, 2011, at 1:38 AM, Stephen Thorne wrote: > On 2011-04-07, Glyph Lefkowitz wrote: >> Because, frankly, Python installation tools REALLY REALLY SHOULD be >> able to install Python files into Python packages. I'm not sure I can >> make any other assertions quite so strongly. I'm pretty su

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Stephen Thorne
On 2011-04-07, Glyph Lefkowitz wrote: > Because, frankly, Python installation tools REALLY REALLY SHOULD be > able to install Python files into Python packages. I'm not sure I can > make any other assertions quite so strongly. I'm pretty sure that > this is a problem that more than one project is

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Tim Allen
On Thu, Apr 07, 2011 at 12:54:45AM -0400, Glyph Lefkowitz wrote: > If we invent our own file extension which has to be separately > installed, we have to teach distutils, and setuptools, and distribute, > and pip, and distutils2, and 'packaging' (as I'm sure that will > eventually be incompatible w

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Glyph Lefkowitz
On Apr 7, 2011, at 12:06 AM, Phil Christensen wrote: > On Apr 6, 2011, at 8:35 PM, Stephen Thorne wrote: >> For your consideration, and (constructive) critcism, here is a twisted >> plugin that is nearly identical to 6 that I have running in >> production: > [snip] >> serviceMaker = ExampleServ

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Glyph Lefkowitz
On Apr 7, 2011, at 12:31 AM, Tim Allen wrote: > On Thu, Apr 07, 2011 at 10:35:18AM +1000, Stephen Thorne wrote: >> So the goal of my post to this mailing list is: >> >> * I would like glyph's goal of having less arbitary code executed at >> twistd launch time to become a realisation, >> >> * I

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Tim Allen
On Thu, Apr 07, 2011 at 10:35:18AM +1000, Stephen Thorne wrote: > So the goal of my post to this mailing list is: > > * I would like glyph's goal of having less arbitary code executed at > twistd launch time to become a realisation, > > * I would like the process of creating a twisted plugin to

Re: [Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Phil Christensen
On Apr 6, 2011, at 8:35 PM, Stephen Thorne wrote: > For your consideration, and (constructive) critcism, here is a twisted > plugin that is nearly identical to 6 that I have running in > production: [snip] > serviceMaker = ExampleServiceMaker() > > From these 19 lines of code there are 4 things t

[Twisted-Python] Twisted Plugins - Implementation Discussion

2011-04-06 Thread Stephen Thorne
G'day, So Glyph and I had a discussion about the architecture and implementation of plugins on IRC this week, I raised some issues that I've seen with implementing plugins in that discussion, and he said that I should take the discussion to the list because IRC wasn't the right place for it. Firs

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread anatoly techtonik
On Wed, Apr 6, 2011 at 3:58 PM, wrote: > > As others have pointed out, this is referring to an old feature which is > being phased out.  It would be great if you could file a ticket for > cleaning up the docs; they mostly shouldn't talk about application > persistence (certainly not in the UDP se

Re: [Twisted-Python] Instrumenting Reactors

2011-04-06 Thread anatoly techtonik
On Wed, Apr 6, 2011 at 1:35 PM, Laurens Van Houtven <_...@lvh.cc> wrote: > Hi! > > > No guarantee it's an optimal solution, but when I wanted to inspect a > running process (which sounds like what you're doing), I had a pretty good > time with manhole. (If you're not familiar with that: it's in Twi

Re: [Twisted-Python] Instrumenting Reactors

2011-04-06 Thread Phil Mayers
On 06/04/11 13:19, Itamar Turner-Trauring wrote: > On Wed, 2011-04-06 at 10:55 +0100, Paul Thomas wrote: >> I have a reactor which is getting busier over time and I'd like to >> find out where the cycles are going. Using the profiler isn't really >> practical on a server running for days, so I'd li

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread Laurens Van Houtven
Urgh, thanks gmail -- that was supposed to be to mithrandi, not to the entire mailing list. My apologies. cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread Laurens Van Houtven
Cool, thanks for the link to Eridanus! I hadn't seen it yet :-) cheers lvh ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread exarkun
On 07:16 am, techto...@gmail.com wrote: >Hi again, >> From http://twistedmatrix.com/documents/current/core/howto/udp.html > >"""As you can see, the protocol is registered with the reactor. This >means it >may be persisted if it's added to an application...""" > >What does 'persisted` here mean? A

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread Tristan Seligmann
On Wed, Apr 6, 2011 at 1:32 PM, Laurens Van Houtven <_...@lvh.cc> wrote: > On Wed, Apr 6, 2011 at 1:27 PM, Tim Allen wrote: >> The difference between Pickle and SQLite is that a SQLite database has >> probably had some thought put into its schema, and is much less likely >> to accidentally scoop u

Re: [Twisted-Python] AMP Client disconnecting from server on high memory usage

2011-04-06 Thread Dan-Cristian Bogos
Hey Jean-Paul, For record's sake, I thought I announce here that my issue got fixed. The problem was that I was loading the data in startService method and it looks like there are some timers which prevent the method to block for a longer period of time. Loading the data in __init__ of service.Se

Re: [Twisted-Python] Instrumenting Reactors

2011-04-06 Thread Itamar Turner-Trauring
On Wed, 2011-04-06 at 10:55 +0100, Paul Thomas wrote: > I have a reactor which is getting busier over time and I'd like to > find out where the cycles are going. Using the profiler isn't really > practical on a server running for days, so I'd like to instrument the > reactor to show me which I/O ev

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread Laurens Van Houtven
On Wed, Apr 6, 2011 at 1:27 PM, Tim Allen wrote: > On Wed, Apr 06, 2011 at 12:41:08PM +0200, Laurens Van Houtven wrote: > > Whoa hang on. Without trying to hijack the thread, this is the entire > > premise of infobarb, the IRC bot I'm building for #python-*, except > > s/pickle/sqlite/, so if thi

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread Tim Allen
On Wed, Apr 06, 2011 at 12:41:08PM +0200, Laurens Van Houtven wrote: > Whoa hang on. Without trying to hijack the thread, this is the entire > premise of infobarb, the IRC bot I'm building for #python-*, except > s/pickle/sqlite/, so if this is a horrible idea I'd like to know before I > build it.

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread Laurens Van Houtven
On Wed, Apr 6, 2011 at 9:30 AM, Tim Allen wrote: > On Wed, Apr 06, 2011 at 10:16:03AM +0300, anatoly techtonik wrote: > > From http://twistedmatrix.com/documents/current/core/howto/udp.html > > > > """As you can see, the protocol is registered with the reactor. This > means it > > may be persiste

Re: [Twisted-Python] Instrumenting Reactors

2011-04-06 Thread Laurens Van Houtven
Hi! No guarantee it's an optimal solution, but when I wanted to inspect a running process (which sounds like what you're doing), I had a pretty good time with manhole. (If you're not familiar with that: it's in Twisted Conch, and it's basically just a way to SSH in to a running box and get a REPL

[Twisted-Python] Instrumenting Reactors

2011-04-06 Thread Paul Thomas
I have a reactor which is getting busier over time and I'd like to find out where the cycles are going. Using the profiler isn't really practical on a server running for days, so I'd like to instrument the reactor to show me which I/O events are taking the longest to deal with. Should I just h

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread Tim Allen
On Wed, Apr 06, 2011 at 10:16:03AM +0300, anatoly techtonik wrote: > From http://twistedmatrix.com/documents/current/core/howto/udp.html > > """As you can see, the protocol is registered with the reactor. This means it > may be persisted if it's added to an application...""" > > What does 'persis

Re: [Twisted-Python] Persisted protocol?

2011-04-06 Thread Stephen Thorne
On 2011-04-06, anatoly techtonik wrote: > From http://twistedmatrix.com/documents/current/core/howto/udp.html > > """As you can see, the protocol is registered with the reactor. This means it > may be persisted if it's added to an application...""" > > What does 'persisted` here mean? It sounds

[Twisted-Python] Persisted protocol?

2011-04-06 Thread anatoly techtonik
Hi again, >From http://twistedmatrix.com/documents/current/core/howto/udp.html """As you can see, the protocol is registered with the reactor. This means it may be persisted if it's added to an application...""" What does 'persisted` here mean? -- anatoly t. __