I do not quite know the most recent status of txws.py, whether it is being
merged to Twisted or not.
But I noticed that it has it has two incompatibilities with Python 2.4:
1. from hashlib import md5, sha1
2. protocol = "wss" if self.isSecure() else "ws"
After fixing those, it w
Maybe you could commit that?
try:
#python 2.6+
from hashlib import md5, sha1
except ImportError:
import md5
from sha import sha as sha1
protocol = "self.isSecure() and wss" or "ws"
That would help to people with 2.4!
From: Corbin Simpson [mailto:mostawesom
We need to limit the new connection rate from a given IP, such that for
example 10.10.1.10 cannot connect more often than 10 times per minute.
This is a high-volume TCP Twisted server, with about 500 new distinct IP
connections per second in the normal state (and we run 8 of these on each 8
core s
I have been operating a busy twisted TCP server for quite some time (1
year+).
Yesterday, it strangely crashed under peak load for that day (it has been
under more load in the past).
# grep segfault /var/log/messages
Mar 20 19:12:15 serv2 kernel: [17687209.144548] twistd[10701]: segfault at
y, March 22, 2009 8:10 AM
> To: Twisted general discussion
> Subject: Re: [Twisted-Python] strange server crash
>
> On Sun, Mar 22, 2009 at 10:32 AM, Itamar Shtull-Trauring
> wrote:
> > On Sat, 2009-03-21 at 20:02 -0700, Alec Matusis wrote:
> >
> >>
I have tested for bad RAM when the server was installed 3 month ago, I did a
memtest that run for a day.
This is an 8 core server, and I run one twistd process per core, and
surprisingly, only one crashed out of 8. Would you think that the effect of bad
RAM would be confined to just once server
This server crashed again today, again during maximum load for the day.
This time, no errors in the twisted log, and not even a segfault message in
/var/log/messages : the pid simply ceased to exist.
Once again, this machine runs 8 twisted servers, but this one is slightly
different from the othe
eved the system even log (SEL) list from the BMC controller, and are no
errors whatsoever.
This looks like a Python error or something very basic. It started happening
after I slightly changed the code for this particular server.
-Original Message-
From: Alec Matusis [mailto:matu...@yahoo
One data point:
I have a server that processes about 20 UDP multicast packets per second.
This server does many other things, but the load from those 20 UDP pps is
negligible- it uses 0% CPU with just that (Xeon L5420)
> -Original Message-
> From: twisted-python-boun...@twistedmatrix.com [
I think 1024 on most linux kernels (2.4 and 2.6) for select, you need to go to
poll/epoll if you want more, or recompile your kernel.
> -Original Message-
> From: twisted-python-boun...@twistedmatrix.com [mailto:twisted-python-
> boun...@twistedmatrix.com] On Behalf Of Drew Smathers
> Sen
> I would say a function of the operating system - or specifically
> ulimit on a *nix system.
I do not think this is correct, on linux: ulimit -n is the number of fds that
can be written/read by a process, but the 1024 limit on select is actually
hard-compiled into the kernel and can be changed
equ...@twistedmatrix.com
> >
> > You can reach the person managing the list at
> > twisted-python-ow...@twistedmatrix.com
> >
> > When replying, please edit your Subject line so it is
> Longer answer: Looks like maybe you want to contribute a patch for this
> ticket :)
>
> http://twistedmatrix.com/trac/ticket/2234
I will look at this at some point (really swamped with work). But my question
is: are you going to use python's select.epoll() that was introduced in python
2
Installing 8.2.0:
creating
/web20/encap/python-2.4.4/lib/python2.4/site-packages/Twisted-8.2.0-py2.4-li
nux-x86_64.egg
Extracting Twisted-8.2.0-py2.4-linux-x86_64.egg to
/web20/encap/python-2.4.4/lib/python2.4/site-packages
File
"/web20/encap/python-2.4.4/lib/python2.4/site-packages/Twisted
Sorry never mind: I found that theres a ticket for it, and downloaded a
patch from:
http://twistedmatrix.com/trac/attachment/ticket/3578/compile_error_avoid.pat
ch
From: twisted-python-boun...@twistedmatrix.com
[mailto:twisted-python-boun...@twistedmatrix.com] On Behalf Of Alec Matusis
Sent
We are using twisted.enterprise.adbapi for connecting to MySQL.
In one of our large tables, the primary key reached its maximum value of
2147483647, and INSERTs became impossible.
When I executed the same INSERT from mysql command line manually, it gave:
ERROR 1062 (23000): Duplicate entry '2147
I am running identical twisted servers under high load, one on 2.5.0
(patched for epoll bug) on python 2.4.4 and second one on 8.2.0 on python
2.6.2.
The 8.2/2.6 one leaks memory compared to 2.5/2.4.4 to the point that I am
about to roll back to older Twisted and then to older Python.
Any insight
> Nothing springs to mind, unfortunately. What parts of Twisted does the
> app use? Maybe narrowing the field a bit will help jog someone's
> memory...
Nothing special:
Epoll reactor and twisted.enterprise.adbapi
Will keep you guys posted.
> -Original Message-
> From: twisted-python-
I have been a long time Twisted user, and I do not know Erlang.
I run into this interesting comment in engineering notes on Facebook Chat
scalability:
http://www.facebook.com/note.php?note_id=51412338919&comments
Leif K-Brooks
I'm curious whether you considered using the Python library Twisted
(
> I think we are accounting way too much time to the unfounded
> unaccountable random blathering of a troll :-)
As you can see from the %CPU column, I have my reasons for concern ;) This
is current copy and paste from a node with 2x quad core xeon L5420 @
2.50GHz - 1 twistd process per core.
PS.
>From my experience, the hardest problem in running large-scale Twisted
servers is finding memory leaks, and to a lesser extent debugging
intermittent silent server crashes.
It's not easy to attach a debugger to a production sever if you are looking
for sources of intermittent crashes for exam
009 12:48 PM
> To: Twisted general discussion
> Subject: Re: [Twisted-Python] I found an interisting comment about
> Twisted vs. Erlang
>
> Hi Alec,
>
> On Tue, Sep 29, 2009 at 12:36 PM, Alec Matusis
> wrote:
> > As you can see from the %CPU column, I have my reasons
>It became very painful once
>we started getting CPU bound on 32 core CPU machines. We had to run
multiple instances of our Twisted server implementation
>to saturate the machine. I have since "ported" the same application to
Erlang/OTP. It is about 1/5 the amount of code.
>And it scales 1:1 horiz
I upgraded to 9.0.0 and I am now seeing a new error, not present in 8.2.0 or
earlier:
2010-02-10 17:38:33-0800 [TagProtocol,9794986,68.126.204.104] Unhandled
Error
Traceback (most recent call last):
File
"/usr/local/encap/python-2.6.4/lib/python2.6/site-packages/Twisted-9.0.0-py2
Assuming you are on Linux, you can run
#man tcp
and take a look at the tunable parameters that control TCP protocol
timeouts:
tcp_keepalive_intvl
tcp_keepalive_probes
tcp_keepalive_time
They can be tuned via /proc filesystem.
These changes are system-wide, so you should consider the performanc
I modified a tolerably leaking (about 40MB/day) Twisted server: when the new
code was pushed, the memory leak became catastrophic (about 100MB/hr).
We could tolerate 40MB/day, but the new code needs to be debugged.
First, I reverted to the old version, that is leaking 40MB/day (The leak
rate is ac
pk = prev.keys()
>pk.sort()
>names = objects.keys()
>names.sort()
>for name in names:
> if limit == 0 or objects[name] > limit:
>if not prev.has_key(name):
> prev[name] = objects[name]
>dt = objects[name] - prev[name]
>
gt;names.sort()
>for name in names:
> if limit == 0 or objects[name] > limit:
>if not prev.has_key(name):
> prev[name] = objects[name]
>dt = objects[name] - prev[name]
>if delta or dt != 0:
> print '%0.6d -- %0.6d
uesday 23 February 2010, Alec Matusis wrote:
>
> > When I start the process, both python object sizes and their counts rise
> > proportionally to the numbers of reconnected clients, and then they
> > stabilize after all clients have reconnected.
> > At that mom
until proven otherwise.
>
> Not to bother you with numbers but I usually allocate about 30% of
> overall project time to bring a server into a production ready state,
> meaning uptimes of months/years, no fishy feelings, no performance
> oscillations, predictable caving and recupe
com] On Behalf Of exar...@twistedmatrix.com
> Sent: Thursday, February 25, 2010 4:51 PM
> To: Twisted general discussion
> Subject: Re: [Twisted-Python] debugging a memory leak
>
> On 01:24 am, johann.bo...@densedata.com wrote:
> >Alec Matusis wrote:
> >>In desperation of not findi
I had 60 busy tcp servers that ran on Python 2.6/Twisted 10.0 (or Twisted
9.0) for over two years with not problems. After I upgraded to Twisted
12.3/Python 2.7, I started getting the errors below (no application code
changes). It took about two days for the first error to appear on a busy
server u
> If you don't find a ticket for the bug, you should report it. Please be
sure to include enough information to reproduce the issue if you file a new
ticket (or as much information as you have, at least).
> Jean-Paul
I filed http://twistedmatrix.com/trac/ticket/6346
It's hard to do any detailed
> By reading these comments, It looks like recent version of twisted
>may
>suffer from this problem. So what version of twisted should I use? Do
>I
>need to keep using 10.1?
This problem is very frequent on Linux with Python 2.7 / Twisted 12
combination (we had to quickly roll back- cannot oper
ted 10.1 as
I wrote).
How can I check whether twisted is using Pythons epoll or its own in this
combination? And how can I switch between epoll implementations in my app?
Alec
From: Gelin Yan [mailto:dynami...@gmail.com]
Sent: Tuesday, May 28, 2013 6:42 PM
To: Alec Matusis
Subject: Re: [Twis
...@twistedmatrix.com] On Behalf Of Gelin Yan
Sent: Tuesday, May 28, 2013 9:09 PM
To: Twisted general discussion
Subject: Re: [Twisted-Python] any progress for Ticket #6346, #6378?
On Wed, May 29, 2013 at 10:37 AM, Alec Matusis wrote:
Hi Gelin,
> As I know there are two epoll implementations in twis
36 matches
Mail list logo
