Re: [Twisted-Python] New Supported Platforms - Python 3.3 and Python 3.4 on Debian 7

Awesome work, Hawkie! Thanks for going through all nitty gritty details and - for getting everything reviewed/merged;) Cheers, /Tobias > -Ursprüngliche Nachricht- > Von: twisted-python-boun...@twistedmatrix.com [mailto:twisted-python- > boun...@twistedmatrix.com] Im Auftrag von HawkOwl

Re: [Twisted-Python] Exception's Implicit Public API, and Python 3

Put differently, Twisted's "the first one's free" policy applies to upgrading Twisted itself, and /not/ to upgrading Python (or any other dependency). If you upgrade Python and you need to update your code for that, Twisted won't create any additional problems but it won't go out of its way to so

[Twisted-Python] Crossbar.io 0.11.0

Hi all, I am pleased to announce Crossbar.io 0.11.0 https://github.com/crossbario/crossbar#crossbario https://pypi.python.org/pypi/crossbar/0.11.0 Crossbar.io is an open-source WAMP application router that allows to build advanced applications from loosely-coupled components that can talk in

[Twisted-Python] Twisted Web at 630k reqs/sec

Hi, maybe it's of interest how far it's possible to drive Twisted Web on modern multi-core CPUs. Here are 2 data points from measuring: * 627990 HTTP requests/s at 360 us avg latency * 12.6 GB/s HTTP reply traffic This is using 40 Xeon cores and serves HTTP over loopback TCP to 8 threads of

Re: [Twisted-Python] Twisted Web at 630k reqs/sec

Those are some nice numbers, to be sure! Twisted really benefits from running in a pypy vm. I have read that pypy still doesn’t optimize old-style classes as efficiently as new-style classes; I wonder if there would be any meaningful change in the metrics if resource.Resource was a new-style

Re: [Twisted-Python] Twisted Web at 630k reqs/sec

Hi Tobias Have you tried to test crossbar on windows? I heard that stdio Yes, I have tried that;) Crossbar.io supports Windows! performs very bad on windows. I noticed crossbar use pipe to transfer data between process. Only for control/management: that is traffic between the node con

Re: [Twisted-Python] Streaming HTTP

We should then move to an API that is much more like the one used by Go: specifically, that by default all requests/responses are streamed. > Request objects (and, logically, any other object that handles requests/responses, such as Resource) should be extended to have a chunkReceived method >

Re: [Twisted-Python] Twisted Web at 630k reqs/sec

Are you sure IOCP is able to share sockets between process? It would be great if you can share some info about this. I only use No, I am not sure. But this SO reply seems to suggest http://stackoverflow.com/questions/14388706/socket-options-so-reuseaddr-and-so-reuseport-how-do-they

Re: [Twisted-Python] Streaming HTTP

Am 17.11.2015 um 11:49 schrieb Cory Benfield: On 15 Nov 2015, at 10:18, Tobias Oberstein wrote: How does flow-control work with the Go API? How does user code processing chunks received unleash backpressure onto the sender? One caveat: I am not an expert in what Go is doing here, more a

[Twisted-Python] ANN: txaio-etcd

Hi, txaio-etcd is a etcd3 client library for Twisted that is fully asynchronous and pure Python: https://github.com/crossbario/txaio-etcd It also exposes advanced features like watching keys, transactions and leases: https://github.com/crossbario/txaio-etcd/tree/master/examples Releases a

Re: [Twisted-Python] Python3 twistd daemon for Ubuntu 14.04 alternatives

Honestly, in the years I’ve been running Python services of different sizes, I have found that distro-provided system packages – unless you are writing software *for* a distribution – are loaded with so many downsides that they’re almost never worth it. They’re a shortcut and shortcuts usually

Re: [Twisted-Python] Elliptic Curve support

Hi, I do think letting users configure the permissible curves (and their priority) using a public tx API, and exposing all of the curves that the underlying openssl provides is important. FWIW, the german BSI (a state IT security institute that provides recommendations, and certifications) n

Re: [Twisted-Python] txtorcon 0.19.0

congrats on the release! > * big revamp / re-write of the documentation, including the new >Programming Guide: >https://txtorcon.readthedocs.io/en/latest/guide.html The new docs are very nice .. and the txtorcon logo in "big" looks really cool;) __

Re: [Twisted-Python] stop/start client connections with loseConnection in ReconnectingClientFactory

Hopefully at least some of this is convincing :) all and each of them is convincing;) -g _______ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- Tobias Oberstein - phone +4

Re: [Twisted-Python] Drop support for Python 3.4?

Python 3.5 is still the default python3 version in Debian stable, so updating beyond that might make it more complex for Debian users to install Twisted. yeah, that, and: as of today, pypy3 is at python 3.5 so _not_ supporting 3.5 would definitely be a problem for us (crossbar.io) dropping 3.4

Re: [Twisted-Python] stop/start client connections with loseConnection in ReconnectingClientFactory

The Autobahn guys still show ReconnectingClientFactory in their docs, Where did you find that? That would be a doc bug, but in the _docs_, there is no reference to ReconnectingClientFactory We do have some example code using ReconnectingClientFactory though: Yeah, this one: https://github.c

Re: [Twisted-Python] stop/start client connections with loseConnection in ReconnectingClientFactory

Am 02.04.19 um 09:03 schrieb Chris Withers: On 02/04/2019 07:56, Tobias Oberstein wrote: Would be good to get that converted... yeah, agreed. unlikely to happen though (lack of time) unless someone steps up with a PR Okay, but then don't be surprised when people get tripped up

Re: [Twisted-Python] stop/start client connections with loseConnection in ReconnectingClientFactory

Am 02.04.19 um 09:50 schrieb Chris Withers: On 02/04/2019 08:25, Tobias Oberstein wrote: https://github.com/crossbario/autobahn-python/blob/master/examples/twisted/wamp/basic/client_using_clientservice.py Any way to get rid of the ugly duplication between the url in line 69 and the endpoint

Re: [Twisted-Python] Twisted 19.2.0rc2 Release Candidate Announcement

Hi Amber, I did a quick test for crossbar/autobahn (linux, fresh venvs): looks good! cpython 3.7.2 https://gist.github.com/oberstet/c950a07ca8c3d52d405fa2994f1ce3a9 pypy 7.0.0 https://gist.github.com/oberstet/ce0642528a2a71dd451c13681adda3e8 greets, /Tobias Am 27.03.19 um 10:59 schrieb Am

Re: [Twisted-Python] ws(s):// urls and host/port duplication in twisted/autobahn code

Am 10.04.19 um 19:46 schrieb meejah: Chris Withers writes: Not sure I follow, I specify ws:// and things work. What am I missing? (I know the initial protocol is http and that's "upgraded" into websocket, but the url is still ws:// or wss://, right?) What Tobias is getting at, I think, is th

[Twisted-Python] Twisted Conch on v19.2.1: now requires explicit installation of bcrypt

Hi, I'm not sure if it considered a Twisted bug, but Twisted Conch on Twisted 19.2.1 now seems to require explicit installation of bcrypt, which wasn't the case on <19.2.1. Here is our (Crossbar.io) tracking issue: https://github.com/crossbario/crossbar/issues/1555 Not a big deal obviously

Re: [Twisted-Python] Twisted Conch on v19.2.1: now requires explicit installation of bcrypt

Hi Glyph, I'm not sure if it considered a Twisted bug, It's not :). but Twisted Conch on Twisted 19.2.1 now seems to require explicit installation of bcrypt, which wasn't the case on <19.2.1. This is not exactly accurate. What happened here is that your other dependencies /happened/ to le

Re: [Twisted-Python] Twisted tips for designing highly concurrent twisted REST API

ailing list Twisted-Python@twistedmatrix.com <mailto:Twisted-Python@twistedmatrix.com> https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-

[Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

Hi all, rgd Twisted v21.2.0, thanks a lot to all involved in this release! the release, mypy, more py3, .. +1! really appreciated. now, fwiw, just wanted to let you know (sorry, too late) about a regression with https://github.com/crossbario/crossbar that actually originates from https://

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

Hi Craig, thanks for your answer, tips and questions! much appreciated Am 03.03.21 um 02:58 schrieb Craig Rodrigues: On Tue, Mar 2, 2021 at 12:12 PM Tobias Oberstein mailto:tobias.oberst...@gmail.com>> wrote: this fix broke our workaround for the bug now fixed;) I had to rea

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

thon mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python -- Tobias Oberstein - phone +49 176 2375 2055 - tobias.oberst...@crossbario.com Crossbar.io GmbH - Waldstrasse 18 - 91054 Erlangen HRB 15870 - Amtsgericht Fuerth - Geschäftsf

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

on their end.  If initially, say, crossbar and matrix would like to work with us to set up some kind of repeatable pattern we can suggest to others, that would be great. ok, I'll take that to the team: https://github.com/crossbario/crossbar/issues/1867 tldr: we could run our CI in 2 sets, one

Re: [Twisted-Python] Twisted v21.2.0 breaks Crossbar.io

but we have had enough difficulty keeping our CI configuration current based on what cloud provider is falling over this month ;-). Yes, CI seems to be universally one of those things that is conceptually simple but somehow takes hours and hours to maintain. Absolutely agreed, glad that it's

Re: [Twisted-Python] reactor for Linux io_uring

sorry, I missed the beginning of the thread / discussion: are you working on a new, full Twisted reactor on top of io_uring? and async filesystem is "just" one part / additional effort in that? Am 12.03.21 um 05:42 schrieb Ian Haywood: On 11/01/2021 8:26 pm, Glyph wrote: seems somewhat irrelev

Re: [Twisted-Python] reactor for Linux io_uring

interesting. thanks! Am 12.03.21 um 09:59 schrieb Ian Haywood: On 12/03/2021 7:21 pm, Tobias Oberstein wrote: sorry, I missed the beginning of the thread / discussion: are you working on a new, full Twisted reactor on top of io_uring? and async filesystem is "just" one part /

Re: [Twisted-Python] reactor for Linux io_uring

Am 13.03.21 um 01:01 schrieb Ian Haywood: On 12/03/2021 8:29 pm, Tobias Oberstein wrote: since with io_uring the queue/ring to append to never blocks for writes, that buffering in twisted would lead to double buffering (as the ring already buffers) then, io_uring follows a completion IO

[Twisted-Python] Flush socket

For testing implementations of a TCP-based protocol, I am testing if an implementation is agnostic wrt to in what chops in receives octets from the wire ("stream clean"). To do that, I force out octets from the protocol test driver (written in Twisted) in differently sized chops, down to single oc

[Twisted-Python] Flow control

For a test driver for a TCP-based protocol (written in Twisted) I want to send data to a receiver as fast as the receiver can consume data, but not faster. Apparently, a naive loop which just does transport.write() does not work. When I do reactor.select(0) between the writes, it somehow works,

Re: [Twisted-Python] Flush socket

) On 11.08.11 23:13, "Christopher Armstrong" wrote: On Thu, Aug 11, 2011 at 5:06 PM, Tobias Oberstein wrote: For testing implementations of a TCP-based protocol, I am testing if an implementation is agnostic wrt to in what chops in receives octets from the wire ("stream clean&qu

Re: [Twisted-Python] Flow control

ahh, I felt I missed sth;) thx .. I'll look into that On 11.08.11 23:16, "Christopher Armstrong" wrote: On Thu, Aug 11, 2011 at 5:13 PM, Tobias Oberstein wrote: For a test driver for a TCP-based protocol (written in Twisted) I want to send data to a receiver as fast as t

Re: [Twisted-Python] Flush socket

://bugzilla.mozilla.org/show_bug.cgi?id=675961 On 11.08.11 23:13, "Christopher Armstrong" wrote: On Thu, Aug 11, 2011 at 5:06 PM, Tobias Oberstein wrote: For testing implementations of a TCP-based protocol, I am testing if an implementation is agnostic wrt to in what chops in receives octets fro

Re: [Twisted-Python] Flush socket

yes. for FF and Chrome, and other WebSockets clients On 11.08.11 23:40, "Christopher Armstrong" wrote: On Thu, Aug 11, 2011 at 5:18 PM, Tobias Oberstein wrote: well, it does not necessarily correspond, but from what I see on tcpdump with loopback device and on LAN (with TCP NoDela

Re: [Twisted-Python] Flush socket

sorry, my question wasn't clear: I do already disable Nagle by setting TCP NoDelay. And I do a reactor.select(0), which sometimes breaks I guess because of reactor reentry not expected, and it will break when the reactor is not select() based I guess. So the question is: is there an alternative

Re: [Twisted-Python] Flush socket

> > I do already disable Nagle by setting TCP NoDelay. > > > > And I do a reactor.select(0), which sometimes breaks I guess because > > of reactor reentry not expected, and it will break when the reactor is > > not select() based I guess. > > > > So the question is: is there an alternative to react

Re: [Twisted-Python] Flush socket

> This will always be a somewhat unreliable way to test a remote process's > handling of packetization, since there are still two TCP/IP stacks which can > mess around with the data in a variety of ways, but it's as good as you can do > if you want to use normal sockets for this testing. > > A mor

Re: [Twisted-Python] Flush socket

Thanks for the tip, I've gone that route as it was the simplest to realize. Since I had centralized all calls to transport.write() in a wrapper in the code, I didn't had to implement an ITransport .. It seems to "work" (modulo the cautions you iterated) .. and I no longer use reactor internals.

Re: [Twisted-Python] 11.0 in PPA

Well, I'm just a Twisted user, but there is http://pypi.python.org/pypi/Twisted/11.0.0 which I'd guess is trivial to build on Ubuntu (python setup.py install) from the source tarball. I know that's not exactly your question;) > -Ursprüngliche Nachricht- > Von: twisted-python-boun...@tw

[Twisted-Python] RPC design questions

I'm currently making some light-weight RPC/PubSub over WebSockets using Twisted. I have two design-like questions where I'd be happy for advice/options ... The first is definitely Twisted related, the second not strictly, .. Tobias 1) I'd like to chain RPC calls, i.e. self.call(23, "square").

Re: [Twisted-Python] RPC design questions

> > self.call(, ) > > self.call("square", 6).addCallback(self.call, > > "sqrt").addCallback(self.show) > > The easiest way to do this is probably: > > from functools import partial > self.call("square", 6).addCallback(partial(self.call, > "sqrt")).addCallback(self.show) Thanks alot! Did not knew

Re: [Twisted-Python] RPC design questions

anyone using both. Built in stuff. Thanks again, Tobias Von: twisted-python-boun...@twistedmatrix.com [mailto:twisted-python-boun...@twistedmatrix.com] Im Auftrag von Glyph Lefkowitz Gesendet: Dienstag, 23. August 2011 20:30 An: Twisted general discussion Betreff: Re: [Twisted-Python] RPC design quest

[Twisted-Python] Twisted vs jQuery Deferreds (was Re: RPC design questions)

I follow Glyphs advice and only keep the lambda version with standard Twisted Deferred. However, I've got a (I believe) more serious "design" problem. The RPC stuff works in Twisted, but also from JavaScript. So I have 2 audiences (Twisted and JS developers). On JS, I use the Deferreds that co

Re: [Twisted-Python] Twisted vs jQuery Deferreds (was Re: RPC design questions)

Ok. This is most unpleasant. In particular since my options then are: 1) write my own, sane (Twisted like, which I think is the sane way) JS deferred, not use jQuery/CommonJS (which everyone else does) 2) open the door for users running into issues when they do both Twisted & JS/jQuery and don't

Re: [Twisted-Python] Twisted vs jQuery Deferreds (was Re: RPC design questions)

they have then() _and_ pipe(). This is all asking for confusion. - Whatever. Tobias On 25.08.11 00:52, "Tobias Oberstein" wrote: Ok. This is most unpleasant. In particular since my options then are: 1) write my own, sane (Twisted like, which I think is the sane way) JS deferred, no

Re: [Twisted-Python] Twisted vs jQuery Deferreds (was Re: RPC design questions)

This is all messy, personally, I want the Twisted semantics in JS, but want to choose something "mainstream" to make it easy for users to adopt. Well. I'll give me an hour tomorrow to look into Dojo and Mochikit. On 25.08.11 02:01, "Richard Wall" wrote: On 25 Au

Re: [Twisted-Python] Twisted vs jQuery Deferreds (was Re: RPC design questions)

>> Note that as far as I know, MochiKit isn't actively developed anymore. >> Google's Closure has a fork of MochiKit's Deferred's, which seems to >> incorporate some of Dojo's code as well. >> >> >> http://code.google.com/p/closure-library/source/browse/trunk/third_party/clos >> ure/goog/mochik

Re: [Twisted-Python] Twisted vs jQuery Deferreds (was Re: RPC design questions)

I have collected the Deferreds from Nevow MochiKit Closure jQuery (and Twisted) here https://github.com/oberstet/jdeferred/tree/master/others for convenient comparison. I'd be interested in baking a JS Deferred which - replicates the Twisted Deferred in semantics and syntax (API) as closely a

Re: [Twisted-Python] RPC design questions

com [mailto:twisted-python-boun...@twistedmatrix.com] Im Auftrag von Glyph Lefkowitz Gesendet: Dienstag, 23. August 2011 20:30 An: Twisted general discussion Betreff: Re: [Twisted-Python] RPC design questions On Aug 23, 2011, at 10:37 AM, Tobias Oberstein wrote: class AutobahnDeferred

[Twisted-Python] Autobahn WebSockets - Twisted based WS implementation

Announcing Autobahn WebSockets Autobahn WebSockets provides an implementation of the WebSockets protocol which can be used to build WebSockets clients and servers. * supports protocol draft version 10-14 * usable for clients and servers * easy to use basic API * advanced API for frame

Re: [Twisted-Python] Autobahn WebSockets - Twisted based WS implementation

> Hi Tobias - great to hear it, thanks for the announcement! > > I'm curious - is this in any way related to previous work done on Twisted > websockets implementations, such as > ? No, it's not. From a cursory look at the code (didn't knew it existed): Outd

Re: [Twisted-Python] Autobahn WebSockets - Twisted based WS implementation

hon-boun...@twistedmatrix.com] Im Auftrag von Reza Lotun Gesendet: Montag, 12. September 2011 18:10 An: Twisted general discussion Betreff: Re: [Twisted-Python] Autobahn WebSockets - Twisted based WS implementation Hi, On Mon, Sep 12, 2011 at 4:38 PM, Tobias Oberstein mailto:tobias.oberst...@taven

[Twisted-Python] Autobahn WebSockets 0.4.3

New release 0.4.3 of Autobahn WebSockets including: + Hybi-17 support + TLS (secure WebSocket) + lots of new options + improved docs and tutorials + close behavior coverage for testsuite + many small things I forgot Special thanks to Peter Thorson for significant code contributions and fruitful d

Re: [Twisted-Python] Autobahn WebSockets 0.4.3

> Hi, how do I run the unit tests for Autobahn? Trial doesn't work, and there > don't appear to be any instances of unittest's classes in the tree. Please see here http://www.tavendo.de/autobahn/testsuite.html#test_browsers to get started. What do you want to test? Own client, own server? > >

Re: [Twisted-Python] Autobahn WebSockets 0.4.3

> > What do you want to test? Own client, own server? > > I wanted to test Autobahn's internals. Unit tests, not integration tests. The > integration test stuff is pretty cool, too, but I was hoping for tests which > verify Autobahn's behavior internally. You can use Autobahn to test Autobahn of

[Twisted-Python] twistd/tac : when does logging start?

I've got a Twisted application started via a tac file. Within the tac, I also initialize a database and create a db pool for use with adbapi. Now it seems that log messages from the network services are logged, but not from the initial DB setup stuff. When exactly does Twistd start the logging

Re: [Twisted-Python] twistd/tac : when does logging start?

> Hi Tobias, > > You need to do your intialisation (database connections, etc) in a > startService > method of your application, rather than (as you're probably > doing) in the __init__ method. > > Hope this helps. Hi Luke, yeah, that helps;) Thanks! _

[Twisted-Python] adbapi / Postgres : LISTEN/NOTIFY and RealDictCursor

I am using Twisted adbapi, psycopg2 and PostgreSQL which works really great. Now, there are 2 features I'd like to use in this context, which I've been using before without Twisted. I'm missing those. Would be glad for any hints .. If there is work to do to make those features happen, and it's

Re: [Twisted-Python] adbapi / Postgres : LISTEN/NOTIFY and RealDictCursor

after actually looking at the adbapi.py code, it's quite accessible, so I guess doing > dict_cursor = db.cursor(cursor_factory = psycopg2.extras.RealDictCursor) would entail forwarding the desired cursor factory to here https://github.com/powdahound/twisted/blob/master/twisted/enterprise/adbapi

Re: [Twisted-Python] adbapi / Postgres : LISTEN/NOTIFY and RealDictCursor

> > btw: are there plans to get Twisted master repo from svn to git? > > No; the eventual plan is to move to bzr. This is happening very slowly though > :). > > By the way, the official Twisted git mirror _should_ be at > . It looks like nobody's updating tha

Re: [Twisted-Python] adbapi / Postgres : LISTEN/NOTIFY and RealDictCursor

Ok, should anyone else need this, here is working code: class PostgresListenService(twisted.application.service.Service): """ PostgreSQL LISTEN/NOTIFY as Twisted service. http://initd.org/psycopg/docs/advanced.html#asynchronous-notifications """ def __init__(self, dsn, channe

Re: [Twisted-Python] adbapi / Postgres : LISTEN/NOTIFY and RealDictCursor

f: Re: [Twisted-Python] adbapi / Postgres : LISTEN/NOTIFY and RealDictCursor On Nov 3, 2011, at 6:18 PM, Tobias Oberstein wrote: while not self.stopped: if select.select([conn], [], [], self.timeout) == ([], [], []): pass Please note that on high-volume servers, this is

[Twisted-Python] kqueue reactor / ticket #1918

> Is there even a kqueue-based Twisted reactor? > > Yes... ish.  More detail here: . Hi Glyph, first, thanks as always! The pykqueue 1.3 and 2.0 referenced in that ticket seem to be no longer maintained. On the other hand, Python has built-in support

Re: [Twisted-Python] kqueue reactor / ticket #1918

> >Is there any regression/test/stresstest I could run to verify the stuff > >actually works? > > You can run the Twisted test suite: > > trial --reactor kqueue twisted > > (assuming you've hooked up the name "kqueue" to your new version of the > kqueue reactor). > > This is what needs to w

Re: [Twisted-Python] kqueue reactor / ticket #1918

> >This is awesome, BUT... > > > >...would probably have to be distributed outside of Twisted proper (at > >least for the moment), as Twisted supports versions of Python < 2.6, > >where select.kqueue would not exist. > > > >(Maybe this is your plan, but I'm just pointing it out.) > > It would be f

Re: [Twisted-Python] kqueue reactor / ticket #1918

> > if there is anything I can do to clarify (like i.e. running on > > non-virtualized > hardware FreeBSD) let me know. > > Have you run the tests on OS X as well? I'll do that. > > Caveat: as attached to the ticket, there are some errs when running > > trial which don't happen with select reac

Re: [Twisted-Python] kqueue reactor / ticket #1918

> We've said e.g. Windows only supports Python 2.5 or later even when > everything else supports Python 2.4. And we have inlineCallbacks which only > runs on Python 2.5 or later. Saying "kqueue reactor only works on Python 2.6 > and later" seems fine to me. For completeness: there seem to be 2 pac

Re: [Twisted-Python] kqueue reactor / ticket #1918

> > Then, baseline. I've run the trial using select reactor in the > > expectation of finding all tests passed. > > > > This is not the case, there are a couple of failed ones. Those are > > also failed with the kqueue reactor, so I need to know whether I need > > to inspect those or those are prob

Re: [Twisted-Python] kqueue reactor / ticket #1918

ftrag von Glyph Lefkowitz Gesendet: Sonntag, 6. November 2011 22:13 An: Twisted general discussion Betreff: Re: [Twisted-Python] kqueue reactor / ticket #1918 On Nov 5, 2011, at 1:46 PM, Tobias Oberstein wrote: Then, baseline. I've run the trial using select reactor in the expectation of finding all

Re: [Twisted-Python] Twisted Project Jobs Volunteer

> But the whole point of having a Git mirror (and a presence on Github, which I > think is the next step) is to attract new developers to this community. An official presence on Github would be definitely desirable I think, and even more than having a Git mirror technically. Not only attracting

[Twisted-Python] data dispatch on massive connection counts

I am testing a WebSocket based pubsub system .. I have 2 questions .. any hints welcome! Environment: + FreeBSD 8.2 i386 + Python 2.7.2 32-bit + Twisted Trunk + new kqueue() reactor The FreeBSD is tuned for massive connection numbers. I can connect 50k WS clients. Fine. 1) Massive dis

Re: [Twisted-Python] data dispatch on massive connection counts

> Besides callLater, a higher level construct to do this is > twisted.internet.task.cooperate(). > https://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2010-cooperative- > multitasking-with-twisted-getting-things-done-concurrently-11-3352182 is a > presentation dreid did, which should really be tur

Re: [Twisted-Python] data dispatch on massive connection counts

> > 2) Too many files. > > Is there another limit specifically for files, and/or something tunable in > Python/Twisted? > > That's definitely not a limit on Python or Twisted. That's an OS limit. > Are you opening lots of files in addition to your sockets? Python open() will bail out at 32768 ope

Re: [Twisted-Python] data dispatch on massive connection counts

> What does resource.getrlimit(resource.RLIMIT_OFILE) return? > > The Python docs say: > > resource.RLIMIT_OFILE > The BSD name for RLIMIT_NOFILE. It's not there .. neither 2.6 nor 2.7 [autobahn@autobahnhub ~/AutobahnHub/service]$ python Python 2.7.1 (r271:86832, Dec 13 2010, 15:52:15) [GCC 4.

Re: [Twisted-Python] data dispatch on massive connection counts

> That's definitely not a limit on Python or Twisted. That's an OS limit. > Are you opening lots of files in addition to your sockets? As it turns out, you are right, and the full answer is most unpleasant: This http://www.freebsd.org/cgi/query-pr.cgi?pr=148581&cat= bug in FreeBSD libc, which i

Re: [Twisted-Python] data dispatch on massive connection counts

> > I was told that the new Python 3 IO system does not use fopen(), > > however Twisted is not yet there on Python 3, right? > > If you are wild, you can try https://bitbucket.org/pitrou/t3k/ (at least as an > experiment :-)) I am .. but not thaaat much;) > > > There is a backport of that new

Re: [Twisted-Python] data dispatch on massive connection counts

> > Well, this is all absolutely sad. Now we (nearly) have new kqueue, it > > does fly, but I can't break above 32k anyway .. > > Epoll on Linux? what is linux? ;) with the monkey patch to use new IO, I can use the Little Daemon flying on kqueue .. so problem solved. __

Re: [Twisted-Python] data dispatch on massive connection counts

> > and the fact that Python uses fopen() from libc, not open() from Posix > > I know you've solved it know, but just to check, you are of course aware of > os.open (which maps directly to Posix open) Yes, thanks, I am aware. However, i.e. Twisted Web uses open(), not os.open(): twisted.python.

[Twisted-Python] reactor.callLater question

2 short questions .. please consider: a) reactor.callLater(0, fun1) reactor.callLater(0, fun2) Is that allowed .. delay == 0? It seems to work .. but I want to be sure. b) reactor.callLater(T, fun1) reactor.callLater(T, fun2) Is the call _order_ first fun1, then fun2 _guaranteed_? For every T,

Re: [Twisted-Python] Twisted 11.1.0 Released

> Highlights of the 185 tickets closed include: Awesome! Haven't been around on the Twisted homepage for a while, but the release made me to, so I like to add this (not release related): The new homepage is really nice and much more inviting! ___ Twi

[Twisted-Python] Twisted FTP: Data must not be unicode

Should I file a bug? If so, any guidelines what to do? --- --- File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.1.0_r33225-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/defer.py", line 545, in _runCallbacks current.result = callback(curre

Re: [Twisted-Python] Twisted FTP: Data must not be unicode

> Rather than implementing an hard-coded conversion at lower level of FTP > protocol implementation, you could write explicit unicode encoders to > bytestream before making a call to FTP API. > > By doing so you can have an FTP protocol, that beside UTF could also handle > other encodings. I'm no

Re: [Twisted-Python] Twisted FTP: Data must not be unicode

> An even more correct solution would be for `line` to have been encoded > properly already before it was passed to `sendLine`. Where did the data > come from, and why wasn't it encoded already? The data is coming from the FTP directory listing formatting function ftp.DTP. _formatOneListResponse

Re: [Twisted-Python] Twisted FTP: Data must not be unicode

> > Cool, that seems much more reasonable. It would be great if you could > > file a ticket for this. Thanks! http://twistedmatrix.com/trac/ticket/5411 > The source of this evil is FTPAnonymousShell.list as this is the entry point > for > the file name. > FTPAnonymousShell.list Yep, I see. T

[Twisted-Python] Autobahn WebSockets 0.4.10

Hi, Autobahn WebSockets 0.4.10 is done: + various improvements to RPC / PubSub + RFC 6455 support + kqueue support on BSD + various smaller issues fixed + demos updated, docs improved Special thanks to Peter Thorson for RFC6455 changes and test cases. Cheers, Tobias Project Link: http://www.t

[Twisted-Python] Review of kqueue reactor patch (ticket 1918)

Hi all, the latest patch contained in http://twistedmatrix.com/trac/ticket/1918 is - after some iterations - now in a "ready for final review state" for some weeks now. I'd like to ask if someone has time to review it so it can be finally merged and the issue closed .. after 6 years;) Cheers

[Twisted-Python] pushing out same message on 100k TCPs

Hi there, what is the most efficient/performant way of doing the following? I have a short message prepared .. say a string of 100 octets. I want to push out that _same_ string on 100k connected TCPs (on a server). == My thinking was: ideally, the 100 bytes would be transferred to kernel/NIC sp

Re: [Twisted-Python] pushing out same message on 100k TCPs

> > what is the most efficient/performant way of doing the following? > > > > I have a short message prepared .. say a string of 100 octets. > > I want to push out that _same_ string on 100k connected TCPs (on a server). > > > > == > > > > My thinking was: ideally, the 100 bytes would be transferre

Re: [Twisted-Python] pushing out same message on 100k TCPs

> Not to mention the fact that inevitably, you probably are going to want some > security on those connections, which means TLS, which means individual crypto > connections. If there is a need for encryption, then yes, .. but it's not always needed > > I believe the best model for this kind of h

Re: [Twisted-Python] pushing out same message on 100k TCPs

> For what it's worth, real IP multicast is quite commonly used for distributing > short messages to many clients in realtime in some closed networks, such as > financial trading systems. With good network equipment that can handle low- > or zero-loss timely delivery, it does work very well. Empha

Re: [Twisted-Python] pushing out same message on 100k TCPs

> Now, as I understand it, sendfile() will perform zero-copy IO; since the > contents > of the file will undoubtedly be in the page cache, it should in theory DMA the > data straight from the (single copy of the) data in RAM to the NIC buffers. > > It should also handle refcounting for you - you

[Twisted-Python] connectionMade, TLS and DoS protection timeouts

I was wondering how I could protect a Twisted server from evil clients initiating, but never completing a TLS handshake. connectionMade is only called when the TLS handshake has completed, right? When doing listenSSL, is there a hook which is fired right after the TCP handshake is complete, bef

Re: [Twisted-Python] connectionMade, TLS and DoS protection timeouts

> > I was wondering how I could protect a Twisted server from evil clients > > initiating, but never completing a TLS handshake. > > > > connectionMade is only called when the TLS handshake has completed, right? > > > > When doing listenSSL, is there a hook which is fired right after the > > TCP ha

Re: [Twisted-Python] connectionMade, TLS and DoS protection timeouts

> >I was wondering how I could protect a Twisted server from evil clients > >initiating, but never completing a TLS handshake. > > > >connectionMade is only called when the TLS handshake has completed, > >right? > > Actually, this is not right. It's sort of a wart, but connectionMade is > called

[Twisted-Python] Twisted/PyPy rocks!

Just wanted to say: PyPy running Twisted rocks! Works just out of the box: PyPy, Twisted + Autobahn. Speedups of up to multiple times .. without a single code change. Btw: * Tornado seems to be unstable on PyPy .. at least in my testing (sporadic segfaults) * have been able to outperform node.j

Re: [Twisted-Python] Twisted/PyPy rocks!

> Fixing these failing tests, or working on getting the dependencies working on > PyPy, would be a great way to contribute to both Twisted and PyPy. :) > > Jean-Paul It's funny: kqueue seems to be chasing me;) I've got PyPy release built from source on FreeBSD9, and Twisted trunk running complet

[Twisted-Python] Twisted+PyPy+kqueue => rocks^2!

It's flying;) I've added kqueue support to PyPy and got it running with Twisted/PyPy on FreeBSD. https://bitbucket.org/pypy/pypy/pull-request/29/finish-kqueue-support Tested with Twisted trunk ('cause of #1918 kqueue support in Twisted) and hammered with a WebSocket testsuite. Twisted trial not

  1   2   3   >