Re: [Twisted-Python] Where is buildslave configuration?

2011-04-04 Thread anatoly techtonik 
pardon, s/checkout and working copy/checkout version and installed version/

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Where is buildslave configuration?

2011-04-04 Thread anatoly techtonik 
On Mon, Apr 4, 2011 at 7:32 AM,   wrote:
>>
>>I tried to find where build slave configuration is stored to see when
>>7th step was added and remove it
>>http://buildbot.twistedmatrix.com/builders/winxp32-py2.6-msi/builds/356/steps/shell_3
>>perhaps together with the 2nd step
>>http://buildbot.twistedmatrix.com/builders/winxp32-py2.6-msi/builds/356/steps/shell
>>
>>Why? Well, because when I change version in  _version.py, the command
>>`python27 twistd --version` still reports the old value.
>
> I don't understand this.  Why do you want to change this configuration?
> Why do you want to change the version?

I want to stop the version string from being forcefully added to
copyright.py file during distribution build process. It is already
imported from twisted module.

Why? I changed version to check that bin/trial uses the correct
twisted checkout (which it didn't), and to my surprise `bin/trial
--version` still gave me the number before it was edited in both
checkout and working copy. That was weird, because in my working copy
copyright.py doesn't have forceful version override hack and I
couldn't find it there.

--
anatoly t.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [ANNOUNCE] Twisted 11.0.0 Released

2011-04-04 Thread Werner Thie 
On 4/3/11 8:01 PM, Jessica McKellar wrote:
>PAS MAINTENANT CHEF! CHUIS EN TRAIN DE BRANCHER LE REACTEUR
>
> On behalf of Twisted Matrix Laboratories, I am honored to announce the
> release of Twisted 11.0.0.

Kudos to all!

Mahalo nui loa, Werner

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] Creating a hybrid server in Twisted

2011-04-04 Thread Jashank Jeremy 
--text follows this line--

Hi,

I'm trying to write a hybrid server that serves either HTTP or my own
custom protocol depending on how it's addressed (distantly inspired by
the IRC bouncer ZNC).  If it receives a standard GET, POST, HEAD,
etc. it sends HTTP traffic, or if it receives the bareword `CORDELIA',
it switches to Cordelia mode.

HTTP works fine with Twisted.Web.  I can serve pages over HTTP (which is
my desired outcome).  However, I'm not sure how to write the Cordelia
handling code.  I've tried my own custom `render_CORDELIA' function, but
it acts too much like HTTP (ie. it returns a result and terminates the
connection), which is not what I want, as the protocol I've designed
involves establishing a two-way conversation, not a single request and
response pair.

So I'm pretty much stuck in a rut.  I don't want to totally reinvent the
wheel just to be able to protocol-switch; I'd prefer to make use of
existing code from Twisted.  How do I hijack Twisted.Web to add protocol
switching?

Jashank

-- 
Jashank Jeremy
PGP: 0x25A5C309


pgpC4CZyHulel.pgp
Description: PGP signature
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Creating a hybrid server in Twisted

2011-04-04 Thread Andrew Bennetts 
Jashank Jeremy wrote:
[...]
> So I'm pretty much stuck in a rut.  I don't want to totally reinvent the
> wheel just to be able to protocol-switch; I'd prefer to make use of
> existing code from Twisted.  How do I hijack Twisted.Web to add protocol
> switching?

You could override lineReceived along the lines of:

def connectionMade(self):
self.seenFirstLine = False
HTTPChannel.connectionMade(self)

def lineReceived(self, line):
if not self.seenFirstLine and line == 'CORDELIA':
# do your protocol switch; e.g. setRawMode and a flag to
# pass all bytes directly to some other protocol.  If you
# want to be really hackish here you can reassign
# self.__class__…
else:
self.seenFirstLine = True
HTTPChannel.lineReceived(self, line)

Alternatively, you could write a protocol decorator to do much the same
thing (i.e. a Protocol that wraps around the HTTPChannel instance).
There's some infrastructure in twisted.protocols.policies that may help
you write that.

(This sort of thing may make an interesting example to add to the
examples in our docs.  I can imagine it'd be possible to add a
ProtocolSwitchingDecoratorBase or similar to twisted.protocols.policies
to make it easier.  It's not a common requirement, but it is something
that people want to do from time to time.  I know I've done it…)

-Andrew.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Where is buildslave configuration?

2011-04-04 Thread exarkun 

On 07:21 am, techto...@gmail.com wrote:

On Mon, Apr 4, 2011 at 7:32 AM,   wrote:


I tried to find where build slave configuration is stored to see when
7th step was added and remove it
http://buildbot.twistedmatrix.com/builders/winxp32-py2.6-msi/builds/356/steps/shell_3
perhaps together with the 2nd step
http://buildbot.twistedmatrix.com/builders/winxp32-py2.6-msi/builds/356/steps/shell

Why? Well, because when I change version in �_version.py, the command
`python27 twistd --version` still reports the old value.


I don't understand this.  Why do you want to change this 
configuration?

Why do you want to change the version?


I want to stop the version string from being forcefully added to
copyright.py file during distribution build process. It is already
imported from twisted module.

Why? I changed version to check that bin/trial uses the correct
twisted checkout (which it didn't), and to my surprise `bin/trial
--version` still gave me the number before it was edited in both
checkout and working copy. That was weird, because in my working copy
copyright.py doesn't have forceful version override hack and I
couldn't find it there.


Can you back up and explain the connection to the buildbot?  What is 
your ultimate goal, and why does achieving it involve changing Twisted's 
build infrastructure?


The *-msi builders are there so we can build release packages for 
Windows.  That requires setting the version information the way it's 
being set.


Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Where is buildslave configuration?

2011-04-04 Thread Andrew Bennetts 
anatoly techtonik wrote:
[…]
> Why? I changed version to check that bin/trial uses the correct
> twisted checkout (which it didn't), and to my surprise `bin/trial
> --version` still gave me the number before it was edited in both
> checkout and working copy. That was weird, because in my working copy
> copyright.py doesn't have forceful version override hack and I
> couldn't find it there.

So your problem isn't the version string, but that running bin/trial in
your Twisted checkout is finding the wrong version of Twisted?

Also, rather than editing random library files, why not just look at the
output of python -v bin/trial?

-Andrew.


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Where is buildslave configuration?

2011-04-04 Thread anatoly techtonik 
On Mon, Apr 4, 2011 at 4:15 PM, Andrew Bennetts  wrote:
>
> So your problem isn't the version string, but that running bin/trial in
> your Twisted checkout is finding the wrong version of Twisted?

Exactly. But wrong version string was the cause of troubles with debugging it.

> Also, rather than editing random library files, why not just look at the
> output of python -v bin/trial?

I forgot about it, so I've just used python bin/trial --version
..and it showed me the version..
..it was 11.0.0..
..but 11.0.0 is the released version I've installed..
..and the version in the trunk/ (i.e. my working copy)..
..so I decided to modify these versions to differentiate copies somehow..
..because it was 1 minute fix..
..because I forgot about python -v bin/trial..
..which is 1 second test..
..so I've looked up where the script reads version..
..and it appeared that it imports copyright.py..
..which in turn imports version from _version.py..
..so I've modified _version.py in my working copy..
..and got the same 11.0.0 version in bin/trial --version...
.."AHA!", - said I to myself..
..but decided to check ..
.."just in case"..
..because I like to be sure..
..and because you never know who listens to your trac.core.Component..
..or zope.interface..
..or whatever..
..unless you run it..
..so, I've modified _version.py in my installation..
..and..
..guess that?..
..still got the 11.0.0..
..$%#!, - said I to myself..
..[$%#!] * 3, - said I to myself three minutes later..
..or five..
..or more..
..I don't remember..
..but if took less than three minutes, I wouldn't be writing this letter..
..so..
..I compared source trees..
..and found..
..that there is an extra version assignment..
..at the end of copyright.py

So, I wasted some more time (I won't tell how much) to find where is
this funny piece of code in release toolchain that inserts this
version string? After looking at some note that .msi is downloaded
from buildbot, I though "No, that's impossible", but, alas, it was
there. Not even open sourced as it appears.

So, now I am curious why this copyright.py patch is required?

My ultimate goal is to run a complete build of Twisted will all tests
that results in .msi installer on my Vista machine - the machine with
operating system that was in the list of NeededBuildSlaves 30 minutes
ago. =)
--
anatoly t.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] [ANNOUNCE] Twisted 11.0.0 Released

2011-04-04 Thread Glyph Lefkowitz 

On Apr 3, 2011, at 2:01 PM, Jessica McKellar wrote:

> PAS MAINTENANT CHEF! CHUIS EN TRAIN DE BRANCHER LE REACTEUR

MAIS SEULEMENT UNE FOIS PAR POCESSUS 

> Many thanks to Glyph Lefkowitz and Jean-Paul Calderone for
> sanity-checking the pre-releases and release, and to the enthusiastic
> PyCon 2011 sprinters who annihilated dozens of tickets. Thanks to
> *everyone* who contributed tickets, patches, documentation, reviews,
> buildbots, feedback, and assistance to fellow users and developers
> leading up to this release. It is truly a group effort.
> 
>
> 
> is a testament to how much work was done in March alone.

... let's not forget to thank Jessica as well for taking the initiative to get 
this release out the door, volunteering when we were badly in need of a release 
manager (because when aren't we, really) for a relatively quick and efficiently 
run release cycle.  It's great to get new people involved in the process, 
doubly so when their execution of said process is flawless :).

So: thanks!

-glyph___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Where is buildslave configuration?

2011-04-04 Thread exarkun 
On 01:59 pm, techto...@gmail.com wrote:
>On Mon, Apr 4, 2011 at 4:15 PM, Andrew Bennetts  
>wrote:
>>
>>So your problem isn't the version string, but that running bin/trial 
>>in
>>your Twisted checkout is finding the wrong version of Twisted?
>
>Exactly. But wrong version string was the cause of troubles with 
>debugging it.
>>Also, rather than editing random library files, why not just look at 
>>the
>>output of python -v bin/trial?
>
>I forgot about it, so I've just used python bin/trial --version
>..and it showed me the version..
>..it was 11.0.0..
>..but 11.0.0 is the released version I've installed..
>..and the version in the trunk/ (i.e. my working copy)..
>..so I decided to modify these versions to differentiate copies 
>somehow..
>..because it was 1 minute fix..
>..because I forgot about python -v bin/trial..
>..which is 1 second test..
>..so I've looked up where the script reads version..
>..and it appeared that it imports copyright.py..
>..which in turn imports version from _version.py..
>..so I've modified _version.py in my working copy..
>..and got the same 11.0.0 version in bin/trial --version...
>.."AHA!", - said I to myself..
>..but decided to check ..
>.."just in case"..
>..because I like to be sure..
>..and because you never know who listens to your trac.core.Component..
>..or zope.interface..
>..or whatever..
>..unless you run it..
>..so, I've modified _version.py in my installation..
>..and..
>..guess that?..
>..still got the 11.0.0..
>..$%#!, - said I to myself..
>..[$%#!] * 3, - said I to myself three minutes later..
>..or five..
>..or more..
>..I don't remember..
>..but if took less than three minutes, I wouldn't be writing this 
>letter..
>..so..
>..I compared source trees..
>..and found..
>..that there is an extra version assignment..
>..at the end of copyright.py
>
>So, I wasted some more time (I won't tell how much) to find where is
>this funny piece of code in release toolchain that inserts this
>version string? After looking at some note that .msi is downloaded
>from buildbot, I though "No, that's impossible", but, alas, it was
>there. Not even open sourced as it appears.
>
>So, now I am curious why this copyright.py patch is required?

It's required because of restrictions on what version you are allowed to 
use when building an MSI.  These rules are codified in 
distutils.version.StrictVersion:

>>> from distutils.version import StrictVersion
>>> from twisted import __version__
>>> StrictVersion(__version__)
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.6/distutils/version.py", line 40, in 
__init__
self.parse(vstring)
  File "/usr/lib/python2.6/distutils/version.py", line 107, in parse
raise ValueError, "invalid version number '%s'" % vstring
ValueError: invalid version number '11.0.0+r31541'
>>> StrictVersion('11.0.0pre1')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.6/distutils/version.py", line 40, in 
__init__
self.parse(vstring)
  File "/usr/lib/python2.6/distutils/version.py", line 107, in parse
raise ValueError, "invalid version number '%s'" % vstring
ValueError: invalid version number '11.0.0pre1'

The build step is there to ensure the version is something bdist_msi 
will accept.  It's an unintentional side-effect that the change survives 
all the way through to the built page.
>My ultimate goal is to run a complete build of Twisted will all tests
>that results in .msi installer on my Vista machine - the machine with
>operating system that was in the list of NeededBuildSlaves 30 minutes
>ago. =)

Coincidentally, I noticed that page was somewhat outdated and updated 
it.  If someone would *like* to contribute Vista slaves, that's 
completely fine, but I think the Twisted project will be happy enough if 
we have to skip over Vista and just pay attention to Windows 7.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Where is buildslave configuration?

2011-04-04 Thread anatoly techtonik 
On Mon, Apr 4, 2011 at 10:55 PM,   wrote:
>>
>>So, now I am curious why this copyright.py patch is required?
>
> It's required because of restrictions on what version you are allowed to
> use when building an MSI.  These rules are codified in
> distutils.version.StrictVersion:
>
>    >>> from distutils.version import StrictVersion
>    >>> from twisted import __version__
>    >>> StrictVersion(__version__)
>    Traceback (most recent call last):
>      File "", line 1, in 
>      File "/usr/lib/python2.6/distutils/version.py", line 40, in
> __init__
>        self.parse(vstring)
>      File "/usr/lib/python2.6/distutils/version.py", line 107, in parse
>        raise ValueError, "invalid version number '%s'" % vstring
>    ValueError: invalid version number '11.0.0+r31541'
>    >>> StrictVersion('11.0.0pre1')
>    Traceback (most recent call last):
>      File "", line 1, in 
>      File "/usr/lib/python2.6/distutils/version.py", line 40, in
> __init__
>        self.parse(vstring)
>      File "/usr/lib/python2.6/distutils/version.py", line 107, in parse
>        raise ValueError, "invalid version number '%s'" % vstring
>    ValueError: invalid version number '11.0.0pre1'
>
> The build step is there to ensure the version is something bdist_msi
> will accept.  It's an unintentional side-effect that the change survives
> all the way through to the built page.

I've tried to do this directly in setup.py so that local .msi builds
could work too.
http://twistedmatrix.com/trac/ticket/5024

-- 
anatoly t.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Where is buildslave configuration?

2011-04-04 Thread Glyph Lefkowitz 

On Apr 4, 2011, at 5:00 PM, anatoly techtonik wrote:

> I've tried to do this directly in setup.py so that local .msi builds
> could work too.
> http://twistedmatrix.com/trac/ticket/5024

Already reviewed and back to you.  Thanks!___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] @inlinecallbacks and AlreadyCalledError in test cases

2011-04-04 Thread Brad Milne 
Hi all

I have recently started switching to trial.unittest from python's own. The
trouble I'm experiencing is when a timeout occurs in my test, it errbacks().
Then the @inlineCallbacks decorator sees the error and errbacks(). But then
a second @inlineCallback in the chain subsequently sees *that* errback and
tries to errback itself. This results in AlreadyCalledError.

In the test setup, various services are started. These are tracked and then
shutdown again in the teardown. Also, there is some polling that happens as
part of the tests (waiting on db activities, for example). These use
deferLater calls, which are also tracked and torn down in the teardown.

I've tried _suppressAlreadyCalled in various places to no avail.

Below is a simple example that shows the problem.

Many thanks
Brad




from twisted.trial.unittest import TestCase

from twisted.internet import task, reactor

from twisted.internet.defer import inlineCallbacks


#import twisted

#twisted.internet.base.DelayedCall.debug = True

#twisted.internet.defer.setDebugging(True)


class Test1(TestCase):



def setUp(self):

# timeout test in 1 second

self.timeout = 1

self.jobs = []

self.addCleanup(self._tearDown)



def _tearDown(self):

for d in self.jobs:

if d and not d.called:

d.cancel()



@inlineCallbacks

def _waitForChange(self):

# do stuff

d = task.deferLater(reactor, 0.5, lambda : None)

self.jobs.append(d)

yield d

self.jobs.remove(d)

# do more stuff



@inlineCallbacks

def testHere(self):

# do stuff

yield self._waitForChange()

# do more stuff


# This one passes OK

#def testHere(self):

#   return self._waitForChange()


if __name__ == "__main__":

import sys

from twisted.scripts import trial

sys.argv.extend([sys.argv[0]])

trial.run()


___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] @inlinecallbacks and AlreadyCalledError in test cases

2011-04-04 Thread exarkun 
On 01:39 am, brad.mi...@devx.runthered.com wrote:
>Hi all
>
>I have recently started switching to trial.unittest from python's own. 
>The
>trouble I'm experiencing is when a timeout occurs in my test, it 
>errbacks().
>Then the @inlineCallbacks decorator sees the error and errbacks(). But 
>then
>a second @inlineCallback in the chain subsequently sees *that* errback 
>and
>tries to errback itself. This results in AlreadyCalledError.
>
>In the test setup, various services are started. These are tracked and 
>then
>shutdown again in the teardown. Also, there is some polling that 
>happens as
>part of the tests (waiting on db activities, for example). These use
>deferLater calls, which are also tracked and torn down in the teardown.
>
>I've tried _suppressAlreadyCalled in various places to no avail.
>
>Below is a simple example that shows the problem.

I tried running the sample, it completes without error.  I guess that's 
not what you're seeing?  What version of Twisted do you have?

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] @inlinecallbacks and AlreadyCalledError in test cases

2011-04-04 Thread Brad Milne 
On 5 April 2011 14:21,  wrote:

> On 01:39 am, brad.mi...@devx.runthered.com wrote:
> >Hi all
> >
> >I have recently started switching to trial.unittest from python's own.
> >The
> >trouble I'm experiencing is when a timeout occurs in my test, it
> >errbacks().
> >Then the @inlineCallbacks decorator sees the error and errbacks(). But
> >then
> >a second @inlineCallback in the chain subsequently sees *that* errback
> >and
> >tries to errback itself. This results in AlreadyCalledError.
> >
> >In the test setup, various services are started. These are tracked and
> >then
> >shutdown again in the teardown. Also, there is some polling that
> >happens as
> >part of the tests (waiting on db activities, for example). These use
> >deferLater calls, which are also tracked and torn down in the teardown.
> >
> >I've tried _suppressAlreadyCalled in various places to no avail.
> >
> >Below is a simple example that shows the problem.
>
> I tried running the sample, it completes without error.  I guess that's
> not what you're seeing?  What version of Twisted do you have?
>
> Jean-Paul
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>


Hi Jean-Paul

I'm using 10.2.0 on a mac. Here's my interpreter (using buildout):

#!/opt/local/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python
import sys
sys.path[0:0] = [
'/Users/brad/Development/python/workspace/Python-Integration/src',
'/Users/brad/Development/python/eggs/nose-0.11.4-py2.6.egg',
'/Users/brad/Development/python/eggs/MiniMock-1.2.5-py2.6.egg',
'/Users/brad/Development/python/eggs/suds-0.4-py2.6.egg',
'/Users/brad/Development/python/eggs/SOAPpy-0.12.4-py2.6.egg',
'/Users/brad/Development/python/eggs/wstools-0.3-py2.6.egg',
'/Users/brad/Development/python/eggs/fpconst-0.7.2-py2.6.egg',

 
'/Users/brad/Development/python/eggs/Twisted-10.2.0-py2.6-macosx-10.6-x86_64.egg',

 
'/Users/brad/Development/python/eggs/zope.interface-3.6.1-py2.6-macosx-10.6-x86_64.egg',

 '/Users/brad/Development/python/eggs/pymongo-1.9-py2.6-macosx-10.6-x86_64.egg',
'/Users/brad/Development/python/eggs/pytz-2010h-py2.6.egg',
]

The python I have at that location (using MacPorts) is 2.6.6.

Following your email I see that 11.0.0 is released so I tried that and got
the same error:

[ERROR]
Traceback (most recent call last):
  File
"/Users/brad/Development/python/eggs/Twisted-11.0.0-py2.6-macosx-10.6-x86_64.egg/twisted/internet/defer.py",
line 1076, in gotResult
_inlineCallbacks(r, g, deferred)
  File
"/Users/brad/Development/python/eggs/Twisted-11.0.0-py2.6-macosx-10.6-x86_64.egg/twisted/internet/defer.py",
line 1066, in _inlineCallbacks
deferred.errback()
  File
"/Users/brad/Development/python/eggs/Twisted-11.0.0-py2.6-macosx-10.6-x86_64.egg/twisted/internet/defer.py",
line 388, in errback
self._startRunCallbacks(fail)
  File
"/Users/brad/Development/python/eggs/Twisted-11.0.0-py2.6-macosx-10.6-x86_64.egg/twisted/internet/defer.py",
line 448, in _startRunCallbacks
raise AlreadyCalledError
twisted.internet.defer.AlreadyCalledError:


Many thanks
Brad
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python