Re: [Twisted-Python] Serial interface for software program

[anatoly techtonik]

Hi Laurens,

Well, Manhole looks way more advanced than my idea. I just need to
be able to tap into one way application event stream - there is no input
from the other side, so application doesn't need to prepare for that.

So, serial != terminal. There might be in future, but now I am more
interested in accessibility of tap interface rather than its features.

On Mon, May 5, 2014 at 6:42 PM, Laurens Van Houtven <_...@lvh.io> wrote:
> Hi Anatoly,
>
>
> Do you know about manhole? I made a video that also demos axiom, but it
> should show you what manhole does :)
>
> https://www.youtube.com/watch?v=3-UZiO-AnLc
>
> Essentially it's "repl in a running process".
>
>
> hth
> lvh
>
>
> On Mon, May 5, 2014 at 4:43 AM, anatoly techtonik 
> wrote:
>>
>> Hello, network hackers,
>>
>> Recently I've got a Raspberry Pi and a friend showed me its serial
>> interface and helped to setup it. I was stunned. What I saw - you open
>> serial terminal, and you jump in into the middle of boot section. Just
>> open a terminal and you see what's going on inside of R.Pi. Any time.
>> That's was an intro. =)
>>
>> For a long time I wanted the same interface for debugging software
>> programs. When I was hacking on Spyder I found that my human brain is
>> too limited to squeeze details of dynamic of interaction of objects at
>> run-time. Spyder is written in Python with Qt, and it is a parallel
>> application much like any Twisted app is. But I wanted to push the
>> progress, I have to deal with complexity, so I badly needed to be able
>> to draw a video of interaction at run-time. That was the problem I
>> tried to solve.
>>
>> And for the first step I needed a way to connect to Spyder at run-time
>> and see what's going on. I didn't know how to call that way before,
>> but now I have a rather accurate wording - "a serial interface for an
>> application". That is the background for the question I am trying to
>> ask.
>>
>> What is the best way to implement such interface considering the
>> following properties:
>> 1. it needs to be simple (so that you can quickly create python script
>> that reads the info)
>> 2. it needs to be universal (so that script for one app worked for other)
>> 3. it should be asynchronous (connected terminal should not degrade
>> performance)
>> 4. it needs to be reliable (at least detecting missing packets)
>> 5. and cross-platform (and pure python)
>>
>> Why Twisted? For the first I think that this problem is actual for
>> people who are dealing with complexity of dynamic and interconnected
>> systems built with the help of Twisted. For the second I believe it
>> can only be solved or analyzed by people skilled in interprocess and
>> internetwork communications.
>>
>> One button test:
>> 1. program behaves weird
>> 2. user hits the button
>> 3. terminal pops up
>> 4. shows a lot of data for a running program
>>
>> I am interested to know.
>> 1. how the button should discover the program
>> network port seems cool, but what about two parallel programs? what
>> about different programs that provide this interface?
>>
>> 2. how terminal receives the data
>> i see it as line based - one line - one event, everything is text
>>
>> 3. how to make it fast
>> TCP clearly won't here
>>
>> 4. how to send the data
>> what if two threads (or parallel code) produce serial data? who should
>> run the server that serves connected terminals, and how other thread
>> will send message to this server? you can not run two TCP servers on
>> the same port on the app side, so is there an alternative? how to
>> detect missing data in a stream? how to avoid dealing with incomplete
>> packets and low level network details?
>>
>> Thanks. I'd like to get back to hacking on Spyder one day, but this is
>> a stumbling block for me every time I think about it or any program of
>> compared complexity.
>> --
>> anatoly t.
>>
>> ___
>> Twisted-Python mailing list
>> Twisted-Python@twistedmatrix.com
>> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>
>
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>



-- 
anatoly t.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] Serial interface for software program

[anatoly techtonik]

On Mon, May 5, 2014 at 8:24 PM, Itamar Turner-Trauring
 wrote:
> If Spyder *just* uses Qt, it's a single-threaded event loop. There's a 3rd
> party event loop for Twisted that runs on top of Qt (qt4reactor), which
> would allow Twisted and Qt code to run in the same thread.

Spyder executes interpreters (seen by users as consoles) in separate threads:
https://bitbucket.org/spyder-ide/spyderlib/src/5c0af92927a342bb0eb561c1729198d3d96ab74e/spyderlib/interpreter.py?at=default#cl-47
and also monitors them to provide code completion, calltips and other features:
https://bitbucket.org/spyder-ide/spyderlib/src/5c0af92927a342bb0eb561c1729198d3d96ab74e/spyderlib/widgets/externalshell/monitor.py?at=default#cl-116

What I need is a mechanism to discover Spyder instances, attach to them
and see what (possibly prepared events) are occurring. It is important that
events are multiplexed into single stream for detecting race conditions. I
need as simple interface as possible and not Python specific even, that's
why I asked it on the network protocol level (which classic serial interface
operates on).

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] ReactorBuilder doesn't support external reactors ticket reminder (Was: Serial interface for software program)

[anatoly techtonik]

On Tue, May 6, 2014 at 4:02 AM, Itamar Turner-Trauring
 wrote:
> On 05/05/2014 03:06 PM, exar...@twistedmatrix.com wrote:
>>
>> However, I'm not even sure if the GUI reactors *should* be distributed as
>> part of the main Twisted package.  They are already first-class reactors via
>> the plugin system.  Independent releases let them track changes in the GUI
>> library they're each concerned with more quickly than they could if they
>> were part of Twisted itself.
>
>
> Someone remind me to file a "ReactorBuilder doesn't support external
> reactors" ticket tomorrow...

File a "ReactorBuilder doesn't support external reactors" today. )
-- 
anatoly t.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] qt4reactor on PySide (Was: Serial interface for software program)

[anatoly techtonik]

On Mon, May 5, 2014 at 10:06 PM,   wrote:
> On 06:49 pm, sank.dan...@gmail.com wrote:
>>>
>>> There's a 3rd party event loop for Twisted that runs on top of Qt
>>> (qt4reactor), which would
>>> allow Twisted and Qt code to run in the same thread.
>>
>> Speaking of which, why isn't that thing officially part of twisted? Super
>> useful.
>
> The PyQt team insists that the PyQt license is incompatible with Twisted's
> license (this is an intentional simplification, you can read more about the
> issue on PyQt's website if you really care).
>
> Since PySide now exists this is potentially no longer an issue. However, the
> last thing I heard about using PySide to run the Qt reactor is that it
> segfaulted sometimes.
>
> So perhaps the reason is that someone needs to do some work to determine if
> PySide is actually safe to use with the Qt reactor now and, if it is,
> shuffle some code around.

I think Twisted users just need to start using it, have this doc at hand -
https://code.google.com/p/spyderlib/wiki/HowToDebugQtCrash -
and register on https://bugreports.qt-project.org/ if there is a trouble
(or just mail PySide list).

> However, I'm not even sure if the GUI reactors *should* be distributed as
> part of the main Twisted package.  They are already first-class reactors via
> the plugin system.  Independent releases let them track changes in the GUI
> library they're each concerned with more quickly than they could if they
> were part of Twisted itself.
>
> Perhaps there are other reasons, like simplicity of packaging or quality of
> continuous integration testing, that would override those issues and make it
> worth bringing Qt reactor back into Twisted.  Sorting that out is also a
> task that requires someone interested to step up an do some work, though. :)

The question was - why super useful Qt reactor loop is not a part of Twisted.
I think that if it is really useful, it should,
-- 
anatoly t.

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] qt4reactor on PySide (Was: Serial interface for software program)

[exarkun]

On 07:57 am, techto...@gmail.com wrote:

On Mon, May 5, 2014 at 10:06 PM,   wrote:


So perhaps the reason is that someone needs to do some work to 
determine if

PySide is actually safe to use with the Qt reactor now and, if it is,
shuffle some code around.


I think Twisted users just need to start using it, have this doc at 
hand -

https://code.google.com/p/spyderlib/wiki/HowToDebugQtCrash -
and register on https://bugreports.qt-project.org/ if there is a 
trouble

(or just mail PySide list).


By all means, please use it and report bugs in the appropriate place.

Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] doWrite called on a twisted.internet.unix.Port

[Killian De Smedt]

Hello everybody,

I sometimes see the following error logged by a twisted application, it
only happens sporadically and I cannot even reproduce when trying to
re-execute the exact sequence of those failures. So giving an SSCCE is
quite impossible for now (sorry). Given this trace it's also hard to find
what was actually called/executed.

Unhandled Error
Traceback (most recent call last):
  File "/path/to/twisted.zip/twisted/python/log.py", line 88, in
callWithLogger

  File "/path/to/twisted.zip/twisted/python/log.py", line 73, in
callWithContext

  File "/path/to/twisted.zip/twisted/python/context.py", line 118, in
callWithContext

  File "/path/to/twisted.zip/twisted/python/context.py", line 81, in
callWithContext

---  ---
  File "/path/to/twisted.zip/twisted/internet/posixbase.py", line 619, in
_doReadOrWrite

  File "/path/to/twisted.zip/twisted/internet/base.py", line 1117, in
doWrite

exceptions.RuntimeError: doWrite called on a twisted.internet.unix.Port

I cannot reproduce the scenario where this goes wrong, in the application
there is only a single unix socket which is handled as follows:

class cmd_protocol(basic.Int32StringReceiver):
def stringReceived(self, command):
self.factory.callback(command, self.send_response)

def send_response(self, code, message):
self.sendString(response_msg(result_code = code, message =
message).SerializeToString()) //response_msg is a google protobuf class

class cmd_factory(protocol.ServerFactory):
protocol = cmd_protocol
def __init__(self, callback, *args, **kwargs):
self.callback = callback

class cmd_server(object):
def __init__(self, socket_file, callback, delete_previous_socket =
True):
if delete_previous_socket:
try:
os.remove(socket_file)
except OSError:
pass
reactor.listenUNIX(socket_file, cmd_factory(callback))
#@UndefinedVariable

Does anybody have any idea what could trigger the doWrite internally or how
I at least I can start debugging this?

Thank you,

Killian
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

[Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here

[HawkOwl]

Hi everyone,

I’m sure that some of you have been following the past seven or so weeks of 
Twisted 14.0 release shenanigans, and this email hopes to explain what went 
wrong, what we can do better next time, and where we can go from here.

Problem 1: Twisted 14.0.0pre1 had a regression. This was not noticed in the 
prerelease stage because it was not marked as a regression, where the RM does a 
check for open regressions on the milestone.
What we can do better next time: Tickets that are regressions need to be marked 
as regressions and applied to the release milestone. If you think it might be a 
regression - even slightly - mark it as such, and comment that you are not 
sure. It’s easier to find the ticket later and decide it is not actually a 
regression than have to abort a release because it’s come up after a prerelease.

Problem 2: The fix for the regression was not merged into pre1, the release was 
rerolled from trunk. This meant some pyOpenSSL and TLS improvements got into 
the 14.0 release from pre2 onwards, but introduced new regressions.
What we can do better next time: Do not reroll from trunk to get bug fixes - 
merge them into the release branch. 

Problem 3: The fixes for the regressions were finished after some delay, since 
the fixes had to be written and reviewed. This introduced delays into the 14.0 
release cycle.
What we can do better next time: Rather than fix regressions introduced, the 
ticket that introduced them should be reverted.

Problem 4: The fixes for the regressions did not merge cleanly with the release 
branch. Some 35+ tickets were merged between pre1 and the release of the 
regression fix into trunk.
What we can do better next time: Bug fixes should be based off the release 
branch, not trunk. This reduces the likelihood of code churn or unknown 
dependencies causing problems during the merge.

Problem 5: There was mixed communication whether one of the regression fixes 
was to be introduced in 14.0 or in a bug fix release (14.0.1).
What we can do better: If a fix is intended for merging in to a prerelease, it 
should be raised on the mailing list, so that there is more visibility for its 
intentions.

Problem 6: I personally made several mistakes along the way - from screwing up 
svn merges to interpreting the “abort the release and incorporate the bugfix” 
to apply the initial regression fix. Since the TLS changes were topical, I 
decided that having them out ASAP would be better than not.
What we can do better: Improved docs/automation to reduce the margin for RM 
error, and better automation to make a new release to get out important 
features really easy.

These are the major problems which I have identified - I’m sure there’s plenty 
more, and I would like people to list them if I have not - even if they make me 
look like an idiot ;). We can learn from it, I’m sure.

So, this leaves where to from now. I see a few options, with my estimates for 
work and risk that it’ll explode:

1 - Most work, high risk - Work on making the regression fixes merge cleanly 
with 14.0.0pre5. This is big-ish task with room for error, since there was some 
underlying code churn.
2 - Some work, medium risk - Release 14.0.0pre5 as 14.0 final, and I (or 
another RM if I’m no longer trusted ;) ) initiate the 14.1 release immediately.
3 - Least work, highish risk - Scrap 14.0, begin the 14.1 release immediately. 
since 14.0 tags become 14.1 tags, and we just have to hope that there’s no 
regressions in the 39 tickets fixed between pre1 and now. This may introduce 
issues (since 14.0 is an un-release, and there are questions about what this 
does to our deprecation windows).

If I am to be honest, I much prefer option #3, but I would like opinions from 
other developers, before I go causing more problems than I already have :)

Regards,
HawkOwl



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] doWrite called on a twisted.internet.unix.Port

[exarkun]

On 01:47 pm, killia...@gmail.com wrote:

Hello everybody,

I sometimes see the following error logged by a twisted application, it
only happens sporadically and I cannot even reproduce when trying to
re-execute the exact sequence of those failures. So giving an SSCCE is
quite impossible for now (sorry). Given this trace it's also hard to 
find

what was actually called/executed.


I've seen something like this with a somewhat old version of Twisted and 
a custom reactor.  I never tracked down the cause.


What version of Twisted are you using, what platform are you on, and 
what reactor are you using?


Another useful bit of debug information would be to hack up the 
reactor's `addWriter` method to do a check of the argument.  The call 
stack at *that* point (when the argument is a Port) is more interesting 
than the call stack at the point where `doWrite` is called.


Jean-Paul

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here

[Laurens Van Houtven]

Hi!
​

First of all, thank you so much for working on this. Secondly, provided we
don't find any regressions in pre5, option 2 seems the least footgunny to
me :)


hth
lvh
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here

[Hynek Schlawack]

On 7 May 2014, at 16:07, HawkOwl wrote:

> 2 - Some work, medium risk - Release 14.0.0pre5 as 14.0 final,

I’m +1 on this one.  The pre5 has been widely tested and the only real issues 
are some embarrassing but inconsequential typos.  Let’s get this out before we 
introduce *real* problems.

> and I (or another RM if I’m no longer trusted ;) )

Nonsense, you’re doing great work.

> initiate the 14.1 release immediately.

You can try to do a 14.0.1 for all I care but those space errors IMHO don’t 
really warrant a point release (except that it would give us some practice for 
times when we need it as glyph pointed out on IRC).

signature.asc
Description: OpenPGP digital signature
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here

[Itamar Turner-Trauring]

 

A diff between the 14.0.0pre5 branch and trunk suggests merging it into
trunk would do bad things, so I'm not sure it's releasable as is... This
may be operator error on my part though. ___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] doWrite called on a twisted.internet.unix.Port

[Killian De Smedt]

Hi Exarkun,

Thanks for the quick response. I should have specified those things
immediately.
I manually merged the UDP ipv6 branch in the trunk somewhere in august and
used that one, the version number is reported as [twisted, version 13.1.0].
The platform is always centos though the centos version might range from
5.x to 6.x, 32 bit, but most of the time it runs on a centos 5.2
installation (kernel on my working machine is 2.6.18). Python is 2.7.1 .
I use the default reactor which should come down to the epoll one.

I'll try to hack up the addwriter, it shouldn't be that hard to for just
that application.

Thank you,

Killian


On 7 May 2014 16:11,  wrote:

> On 01:47 pm, killia...@gmail.com wrote:
>
>> Hello everybody,
>>
>> I sometimes see the following error logged by a twisted application, it
>> only happens sporadically and I cannot even reproduce when trying to
>> re-execute the exact sequence of those failures. So giving an SSCCE is
>> quite impossible for now (sorry). Given this trace it's also hard to find
>> what was actually called/executed.
>>
>
> I've seen something like this with a somewhat old version of Twisted and a
> custom reactor.  I never tracked down the cause.
>
> What version of Twisted are you using, what platform are you on, and what
> reactor are you using?
>
> Another useful bit of debug information would be to hack up the reactor's
> `addWriter` method to do a check of the argument.  The call stack at *that*
> point (when the argument is a Port) is more interesting than the call stack
> at the point where `doWrite` is called.
>
> Jean-Paul
>
> ___
> Twisted-Python mailing list
> Twisted-Python@twistedmatrix.com
> http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
>



-- 
Killian De Smedt
mobile: +32 486/825 951
mail: killia...@gmail.com
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here

[Glyph Lefkowitz]

On May 7, 2014, at 7:07 AM, HawkOwl  wrote:

> Hi everyone,

Hi HawkOwl,

> I’m sure that some of you have been following the past seven or so weeks of 
> Twisted 14.0 release shenanigans, and this email hopes to explain what went 
> wrong,

Given that there does not appear to be a 14.0 final, shouldn't this be "what is 
still going wrong"?  This is more like a death rattle, not a post mortem ;-).

> what we can do better next time, and where we can go from here.

Thank so much for doing this.  I'm sorry the 14.0 release process has been a 
tough one, and that its toughness has been partially my fault.

However, I'm glad that this has provoked some reflection and discussion.  The 
fact that you've done such a thorough analysis almost makes a challenging 
release cycle worth it :).

> Problem 1: Twisted 14.0.0pre1 had a regression. This was not noticed in the 
> prerelease stage because it was not marked as a regression, where the RM does 
> a check for open regressions on the milestone.

When you say it was "not noticed in the prerelease stage", do you just mean it 
didn't show up before the pre-release was made?

Also, in the future, can you always include specific links to the tickets 
involved in the problems encountered?  I'm not exactly sure which regressions 
we're talking about in pre1.

> What we can do better next time: Tickets that are regressions need to be 
> marked as regressions and applied to the release milestone. If you think it 
> might be a regression - even slightly - mark it as such, and comment that you 
> are not sure. It’s easier to find the ticket later and decide it is not 
> actually a regression than have to abort a release because it’s come up after 
> a prerelease.

At the same time, I feel like I should stress like this, by itself, was not a 
huge problem.  Specifically, rolling a second pre-release is okay.  It's a bit 
unfortunate that the regression was not tagged in advance of the release, but 
discovering issues and fixing them is exactly what the pre-release process is 
for.

> Problem 2: The fix for the regression was not merged into pre1, the release 
> was rerolled from trunk. This meant some pyOpenSSL and TLS improvements got 
> into the 14.0 release from pre2 onwards, but introduced new regressions.
> What we can do better next time: Do not reroll from trunk to get bug fixes - 
> merge them into the release branch. 

Another problem here, that I can take full blame for, was that the 
communication involved was fragmented and not terribly consistent.  HawkOwl 
would ask a question on IRC, I would give an answer, then a couple of hours 
later someone else would give an apparently contradictory answer to a follow-up 
question.  I don't think that we were actually disagreeing all that much, but 
at a number of points, it became a game of telephone.  Also, I'd sometimes ask 
a question about the release process, and someone would tell me something they 
thought HawkOwl had said or a guess as to what might come next, which I took to 
be the actual plan.

Particularly, I was very confused at various points as to whether the next 
prerelease was going to have things backported, which things were going to be 
backported, or whether we were re-rolling from trunk.  I think that, similarly, 
HawkOwl was very confused as to what I _wanted_ to happen.

In the future, when we're communicating about the release process, we should 
probably try harder than usual to have all the discussion in a persistent forum 
so that it's obvious where the state of things is.  Maybe that means the 
mailing list, maybe the release ticket, but IRC has proven to be a particularly 
inappropriate and unreliable channel for this kind of discussion.

If we _do_ have a discussion on IRC, following the precedent that some more 
responsible members of the community have set, and copying a summary or trimmed 
transcript of the relevant conclusions into the ticket or to the list should be 
a requirement.

To get a head start on this, I have put a link to this very discussion on the 
ticket. 

And a final point on communication: on release branches, sensible commit 
messages are particularly important.  On most branches, individual commit 
messages can be a bit less than helpful because they're eventually all bundled 
up into a squash commit (hopefully one day a proper merge commit) with its own 
useful commit message.  That commit message can fill in any gaps left by 
unhelpful individual commits.

On release branches, however, every individual commit has release implications, 
so explaining why things are being done is extra important.  For example, this 
sequence of events is confusing: 
 
.  Which merge is being 
reverted?  (I can kinda guess it's the immediately preceding commit, but...) 
Did a build fail or something?  Which build?  Were some commits mer

Re: [Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here

[Glyph Lefkowitz]

On May 7, 2014, at 7:42 AM, Itamar Turner-Trauring  wrote:
> A diff between the 14.0.0pre5 branch and trunk suggests merging it into trunk 
> would do bad things, so I'm not sure it's releasable as is... This may be 
> operator error on my part though.
> 
This doesn't look significant.

A squash merge in git gives me these conflicts:

Unmerged paths:

both modified:  twisted/internet/_sslverify.py
deleted by them:twisted/topfiles/5190.feature

The conflict in _sslverify.py is trivial, a single hunk that looks like this:

diff --cc twisted/internet/_sslverify.py
index 5fbdc58,0d2e312..000
--- a/twisted/internet/_sslverify.py
+++ b/twisted/internet/_sslverify.py
@@@ -1493,15 -1479,6 +1493,18 @@@ class OpenSSLCertificateOptions(object)
  return ctx
  
  
++<<< HEAD
 +
 +OpenSSLCertificateOptions.__getstate__ = deprecated(
 +Version("Twisted", 14, 1, 0),
 +"a real persistence system")(OpenSSLCertificateOptions.__getstate__)
 +OpenSSLCertificateOptions.__setstate__ = deprecated(
 +Version("Twisted", 14, 1, 0),
 +"a real persistence system")(OpenSSLCertificateOptions.__setstate__)
 +
 +
++===
++>>> origin/releases/release-14.0.0-7039-3
  
  class _OpenSSLECCurve(FancyEqMixin, object):
  """

so simply deleting the conflict markers ought to merge it satisfactorily.  If I 
do so and run the tests they pass.

5190.feature can simply be deleted in trunk.

-glyph

___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] The Twisted 14.0 Release Pre-Post-Mortem, and Where To From Here

[HawkOwl]

On 8 May 2014, at 5:40, Glyph Lefkowitz  wrote:

> 
> On May 7, 2014, at 7:07 AM, HawkOwl  wrote:
> 
>> Hi everyone,
> 
> Hi HawkOwl,
> 
>> I’m sure that some of you have been following the past seven or so weeks of 
>> Twisted 14.0 release shenanigans, and this email hopes to explain what went 
>> wrong,
> 
> Given that there does not appear to be a 14.0 final, shouldn't this be "what 
> is still going wrong"?  This is more like a death rattle, not a post mortem 
> ;-).

Pre-post-mortem! :)

> 
>> what we can do better next time, and where we can go from here.
> 
> Thank so much for doing this.  I'm sorry the 14.0 release process has been a 
> tough one, and that its toughness has been partially my fault.
> 
> However, I'm glad that this has provoked some reflection and discussion.  The 
> fact that you've done such a thorough analysis almost makes a challenging 
> release cycle worth it :).
> 
>> Problem 1: Twisted 14.0.0pre1 had a regression. This was not noticed in the 
>> prerelease stage because it was not marked as a regression, where the RM 
>> does a check for open regressions on the milestone.
> 
> When you say it was "not noticed in the prerelease stage", do you just mean 
> it didn't show up before the pre-release was made?
> 
> Also, in the future, can you always include specific links to the tickets 
> involved in the problems encountered?  I'm not exactly sure which regressions 
> we're talking about in pre1.

This regression was https://twistedmatrix.com/trac/ticket/6926 - ie. that all 
our docs would be wrong.

> 
>> What we can do better next time: Tickets that are regressions need to be 
>> marked as regressions and applied to the release milestone. If you think it 
>> might be a regression - even slightly - mark it as such, and comment that 
>> you are not sure. It’s easier to find the ticket later and decide it is not 
>> actually a regression than have to abort a release because it’s come up 
>> after a prerelease.
> 
> At the same time, I feel like I should stress like this, by itself, was not a 
> huge problem.  Specifically, rolling a second pre-release is okay.  It's a 
> bit unfortunate that the regression was not tagged in advance of the release, 
> but discovering issues and fixing them is exactly what the pre-release 
> process is for.
> 
>> Problem 2: The fix for the regression was not merged into pre1, the release 
>> was rerolled from trunk. This meant some pyOpenSSL and TLS improvements got 
>> into the 14.0 release from pre2 onwards, but introduced new regressions.
>> What we can do better next time: Do not reroll from trunk to get bug fixes - 
>> merge them into the release branch. 
> 
> Another problem here, that I can take full blame for, was that the 
> communication involved was fragmented and not terribly consistent.  HawkOwl 
> would ask a question on IRC, I would give an answer, then a couple of hours 
> later someone else would give an apparently contradictory answer to a 
> follow-up question.  I don't think that we were actually disagreeing all that 
> much, but at a number of points, it became a game of telephone.  Also, I'd 
> sometimes ask a question about the release process, and someone would tell me 
> something they thought HawkOwl had said or a guess as to what might come 
> next, which I took to be the actual plan.
> 
> Particularly, I was very confused at various points as to whether the next 
> prerelease was going to have things backported, which things were going to be 
> backported, or whether we were re-rolling from trunk.  I think that, 
> similarly, HawkOwl was very confused as to what I _wanted_ to happen.
> 
> In the future, when we're communicating about the release process, we should 
> probably try harder than usual to have all the discussion in a persistent 
> forum so that it's obvious where the state of things is.  Maybe that means 
> the mailing list, maybe the release ticket, but IRC has proven to be a 
> particularly inappropriate and unreliable channel for this kind of discussion.
> 
> If we _do_ have a discussion on IRC, following the precedent that some more 
> responsible members of the community have set, and copying a summary or 
> trimmed transcript of the relevant conclusions into the ticket or to the list 
> should be a requirement.
> 
> To get a head start on this, I have put a link to this very discussion on the 
> ticket. 
> 
> And a final point on communication: on release branches, sensible commit 
> messages are particularly important.  On most branches, individual commit 
> messages can be a bit less than helpful because they're eventually all 
> bundled up into a squash commit (hopefully one day a proper merge commit) 
> with its own useful commit message.  That commit message can fill in any gaps 
> left by unhelpful individual commits.
> 
> On release branches, however, every individual commit has release 
> implications, so explaining why things are being don