Re: [Twisted-Python] Pyjamas + web2py + twisted vs Django + Pinax
On 2008.12.19 10:44:49 +0900, Theron D. Huffman wrote: > I'm about to start programming a game in Python. The game will be a web-based > board game with a social aspect, something like http://www.chesspark.com. I > want the game to run in both browsers and as a desktop client. Therefore I'm > wondering what python-based technologies I need. I know that Javascript/AJAX > is necessary for the browser part so I have decided on the following: > > 1. Pyjamas - (for the AJAX browser stuff) > > 2. Pyjamas-desktop - (for the desktop client) > > 3. Web2py (or djanjo, or django+pinax) - for the website on which the game > will run and for database access and social part - login, chat, members, > profiles, game history, etc.) > > 4. Twisted - for the networking and chat functionality. > > 5. Pygame - for most of the game stuff, sprites, inages, etc. (Actually I > don't know if Pygame is necessary for a board-type game.) > > My understanding of the above technologies is still limited and I'm not sure > if I need all of these because some functionality may be overlapped. e.g. I > don't know where Pyjamas leaves off and where web2py/danjo/pinax begins. > > Can someone please give me some advice? That's a fine laundry list of frameworks, but i believe getting all of them to cooperate would be practically impossible. (Note: a framework is a library that's grown to the point where it thinks it owns your whole program. Getting multiple frameworks to play nicely together is usually hard. There are rare exceptions, like the beautiful integration between Twisted and PyGTK, but they're rare.) I know that Django and pygame and Twisted are useful, mature, documented frameworks that actually get used in production by people other than their authors. I don't know about Pyjamas. If putting 100% Python in one end and getting Ajax out the other is really important to you, then you need Pyjamas to work. (Or wait for PyPy and its JavaScript backend to mature from experimental research code to production-quality stuff.) So my advice is to write a simple tic-tac-toe game in Pyjamas, and find out. -- David Ripton drip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Two thumbs up for inotify.py
I just tried the latest inotify.py from dialtone's sandbox, and I like it. My application is a little PyGTK / Cairo recording data scope (closed source) that needs to read lots of little files and plot lots of data points in realtime. I have my own inotify/ctypes wrapper (at the time I wrote it I found two Python inotify wrappers; one was GPL and the other didn't work for me) which works okay in this application. But it's a bit gross because my call to see if there are any inotify events is blocking (with a very short timeout), rather than truly event-based. To try inotify.py, I Twistified my code (easy because it was already event-driven PyGTK), then changed all the inotify calls to use the sandbox module's API rather than mine. The only thing I disliked about the API was having to pass the callbacks are to the watch method, rather than the watch method returning a Deferred that I could addCallback on, but that's just different, not actually hard or confusing. Anyway, it works and my code is a bit smaller and cleaner than it was before and uses a tad less CPU. I hope this ends up in a Twisted release. I see #972 in the tracker, but it's been untouched for a long time. Should I comment there? Thanks. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] [Q] Naming Service for Twisted Servers
On 2009.01.09 23:06:46 -0500, V S P wrote: > What is ZeroConf? Can you please point me to the project page > (I searched for some but did not find something twisted/python > specific). ZeroConf is the generic protocol name, not a particular project. The most popular implementations are Bonjour and Avahi. Both are portable and open source. Macs ship with Bonjour. Avahi seems more popular on Linux. In theory, it's pretty easy to use Avahi from Python. Avahi is a C daemon that always runs and does all the network stuff. It shares information with other programs via DBus. Your Python program can interact with the information using dbus-python. dbus-python features an asynchronous interface that should work great with Twisted. In practice, it all ends in tears because dbus-python is one big Heisenbug. After about 18 months with mostly-working service discovery, we finally gave up on it and wrote our own custom service discovery protocol. (Which works well, but doesn't help you because it's proprietary and IPv6.) Bonjour appears to have Python bindings; maybe you can try those. Or maybe a newer version of dbus-python actually works reliably. But if not, it's not hard to write your own simple service discovery protocol. Create a common multicast group that all your clients listen to, and that all your servers send a packet of service information to every few seconds. Most of the complexity in ZeroConf is about minimizing the combination of latency and network traffic, so if you don't need low latency or don't mind a bit more traffic then a simpler protocol will work fine. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted developers: please donate your time!
On 2009.02.23 13:12:08 -0500, Itamar Shtull-Trauring wrote: > As part of the TSF's fundraising efforts, we are trying to get upfront > donations of time you will spend developing Twisted. It will then get > matched by a donor, if we are successful in getting this grant. So if > you're planning on working on Twisted anyway this year, your work will > count twice as much! > > So, if you're interested, please reply, saying something like "I will > spend two weeks working on Gopher support over the next year." I will spend one week working on IPv6 support over the next year. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted.mail SMTP server question
On 2009.03.12 13:23:50 +0100, Kalman KHIRALY wrote: > I would like to use twisted mail for the following purpose: > Redirect all incoming myn...@mydomain.com mails to a specified gmail address: > myn...@mydomain.com -> myname.mydom...@gmail.com Do you really want to write code for this? That's a one-line .forward file, or (if you want to be fancy) one line in a .procmailrc file. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] official packaging
On 2009.04.02 14:21:14 +1100, Tim Allen wrote: > Esteve Fernandez wrote: > > Anyway, it would be great to have recent packages of Twisted for > > distributions > > that offer some kind of long term support contracts (Ubuntu, RHEL, etc.), > > but > > that tend to get a bit outdated. > > It's worth mentioning that someone has recently stepped up to fix > ticket 1696 after I'd ignored it for Far Too Long, and very soon it > should be possible to build Twisted RPMs for RHEL just by running > "./setup.py bdist_rpm" in a fresh checkout. Thanks Tim. I make RHEL5 rpms for Twisted at work, using "setup.py bdist_rpm" rather than a manually generated spec file. I thought we were just about the only ones who cared about this, so I've never really tried to push the (small) changes upstream. But when your changes land I'll test that they work for us too. An issue with pushing this kind of change to RPM-based distros is that writing .spec files by hand and maintaining them is their core competency, so they mostly think using setup.py bdist_rpm to autogenerate them is weird and unnecessary and wrong. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Automatic test for Twisted
On 2009.05.28 16:41:20 -0400, Drew Smathers wrote: > On Thu, May 28, 2009 at 4:24 PM, Juanjo Conti wrote: > > is there a way to write sort of unittest for Twisted and run them? > > Yes, with twisted.trial: > > http://twistedmatrix.com/projects/core/documentation/howto/testing.html Another option is nose. Google for "nose.twistedtools" -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Application Design help - Concurrent but not Protocols based.
On 2009.06.03 21:55:27 +0530, Senthil Kumaran wrote: > 1) I need to constantly monitor a particular directory for new files. > 2) Whenever a new file is dropped; I read that file and get > information on where to collect data from that is a) another machine b) > machine2-different method c) database. > 3) I collect data from those machines and store it. > > The data is huge and I need the three processes a, b, c to be > non-blocking, and I can just do a function call like do_a(), do_b(), > do_c() to perform them. > > For 1) to constantly monitor a particular directory for new files, I > am doing something like this: > My Question: Can this be designed in way that looking for new files is > also asynchronous activity? If your OS has a way to let you register your interest in particular directories and then notify you when new files appear there, then yes. If you're using Linux then check out the Twisted inotify wrapper that's in dialtone's sandbox. http://twistedmatrix.com/trac/changeset/25717 If you're using something else then it probably has a similar API but it'll be more work because AFAIK nobody's already written the Twisted wrapper for you. Or maybe you can get away with just periodically calling os.listdir from a subthread, using deferToThread. Not technically asynchronous but probably good enough. > Now, after reading the contents, I will have to do a non-blocking call > to fetch data, either using fun_a, fun_b or fun_b. How should I > associate this requirement to deferred/callback pattern? Depends. If it's just a simple cheap Python function that doesn't block then you can just do: deferred1 = reactor.callLater(0, fun_a) deferred1.addCallback(fun_a_callback) deferred1.addErrback(fun_a_errback) If it's a simple function that blocks and can't be changed to not block but doesn't use too much CPU then you can use deferToThread. If it's a piggy enough function that you really want it in a separate process so it can use another CPU core, then write a little Python script that wraps it, and call it using the Twisted process APIs: http://twistedmatrix.com/projects/core/documentation/howto/process.html But just because you can do this in Twisted doesn't mean you necessarily should. If you need an asynchronous main loop then Twisted has really good APIs for dealing with asynchronous main loops. (If you're on Linux and can use inotify then it qualifies.) But if you end up polling the filesystem with os.listdir in one thread, and running your fun_x in other threads, and you're not really doing anything asynchronous, then IMO Twisted won't really add any value. In that case I'd just use Python's threading and Queue modules. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twistd .tac + wxpython
On 2009.08.20 21:00:52 +, Mikhail wrote: > What GUI library would you recommend instead of wx? If clean integration with Twisted is the primary goal, PyGTK. The gtk2reactor integration is fantastic. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Seeking advice on use of Perspective Broker
On 2009.09.17 17:39:17 -0500, Arthur Pemberton wrote: > On Thu, Sep 17, 2009 at 2:09 PM, Brett Sheffield wrote: > > Arthur Pemberton wrote: > >> 1) Is the Twisted Matrix book published by Orielly relevant ie. updated > >> enough? > > > > Yes. I just read it and it has excellent Perspective Broker examples in > > chapters 5 & 6 which got me to a flying start recently. Obviously > > you'll want to read the online docs too. > > > > It doesn't cover twisted.web and nevoew very well because the API was > > still shifting about then, but the rest seems pretty good. It was > > published in 2006. > > I know it was published in 2006, but 3 years can be a long time in the > computing industry, but you've convinced me that its worth a ready. Is PB has changed very little lately. My PB-using project started in 2003. The last time I remember changing something to sync with a Twisted change was in late 2003, when I switched my from using the old cred API to the new cred API. Since then it's been 100% boringly backward-compatible. If you want stable documentation and the most examples and a very strong likelihood that the API won't change underneath you, PB is your best Twisted RPC option. (Other Twisted RPC options include AMP, Foolscap, and txprotobuf. I recommend learning PB first because it has the most examples and docs so it's the easiest to learn. If you end up using one of the others to get better security or portability or whatever, it'll be pretty easy to learn because the Twisted fundamentals carry over.) -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Using reactor with multiple processes
On 2009.09.18 12:01:11 -0500, David Yoakley wrote: > Instead, we implemented a simple framework using twisted > PerspectiveBroker (pb) where we spawn processes with python > subprocess.Popen and use PB remote calls to manage the processes. Its > now working well and is very simple. Sounds great. Thanks for the update. I just want to mention that I mistrust the subprocess module because of lots of personal experience with http://bugs.python.org/issue1731717 . If you see weird intermittent failures to spawn processes, consider switching to reactor.spawnProcess. (IMX such failures are common with Python 2.4, less common with Python 2.5 or 2.6. So if you're not shackled to^W^W using RHEL or some other archaic piece of^W^W^W Enterprise-friendly distribution, you can probably safely ignore this paragraph.) -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Where is TwistedQuotes source code?
On 2009.09.22 22:14:15 -0400, John Crawford wrote: > I'm trying to find the source code for the TwistedQuotes application. > I've looked in every section of the website, and failed to find it, > alas. Is it still there, and if so, where? You can browse it using Trac here: http://twistedmatrix.com/trac/browser/trunk/doc/core/howto/listings/TwistedQuotes Or just checkout the Twisted trunk from svn. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted and wxPython
On 2009.11.05 18:14:57 +, Jon Mills wrote: > Can anybody point me towards a nice up-to-date example of the use of > Twisted with wxPython? > > I assume (hope) there's some way to bind the wx event loop into > Twisted's Reactor? Or do they need to run in separate > threads/processes? wx has multiple event loops so reactor integration isn't as easy as it should be. Longer version of that answer here: http://wiki.wxpython.org/wxPythonAndTwisted Here's an example of doing it with threads: https://pangalactic.us/repo/ampchat -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Launching Twisted client using Process call
On 2009.11.17 07:07:50 +0100, naman jain wrote: > I try to launch it as a Process: > foo = Process( target=twisted_client.main, args=(q,uid,)) > foo.start() > foo.join() > status = q.get() > Launching the client in this manner, is not showing reliable results. You didn't give a long enough example to be certain, but that looks like multiprocessing.Process syntax. The multiprocessing module does not work reliably with Twisted. Neither does the subprocess module. You should use the equivalent Twisted process functionality, shown at http://twistedmatrix.com/projects/core/documentation/howto/process.html There's also https://launchpad.net/ampoule, but for the small example you show, the basic Twisted process stuff should be fine. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted 9.0.0 prerelease 1
On 2009.11.19 22:13:46 -0500, Christopher Armstrong wrote: > Hey guys. I've done an initial cut of Twisted 9. Please check out > these tarballs and tell me whether or not you find any regressions or > other release-related problems. > > The release notes are here: > > http://twistedmatrix.com/users/radix/twisted-prereleases/9.0.0pre1/NEWS.txt > > The tarballs are here: > > http://twistedmatrix.com/users/radix/twisted-prereleases/9.0.0pre1/ > > Please post a message even if you don't have any problems. I just replaced Twisted 8.2.0 with 9.0.0pre1 in my local Git repo at work, and did a full build of our product, and then ran the bits that use Twisted (we use the core and the gtk2 reactor and AMP, on RHEL), and nothing obviously weird or bad happened. Bor-ing! Maybe try fewer unit tests next time, to add more excitement to the upgrade process? -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted 9.0.0 prerelease 1
On 2009.11.20 13:33:24 -0800, David Ripton wrote: > On 2009.11.19 22:13:46 -0500, Christopher Armstrong wrote: > > Hey guys. I've done an initial cut of Twisted 9. Please check out > > these tarballs and tell me whether or not you find any regressions or > > other release-related problems. > > > > The release notes are here: > > > > http://twistedmatrix.com/users/radix/twisted-prereleases/9.0.0pre1/NEWS.txt > > > > The tarballs are here: > > > > http://twistedmatrix.com/users/radix/twisted-prereleases/9.0.0pre1/ > > > > Please post a message even if you don't have any problems. > > I just replaced Twisted 8.2.0 with 9.0.0pre1 in my local Git repo at > work, and did a full build of our product, and then ran the bits that > use Twisted (we use the core and the gtk2 reactor and AMP, on RHEL), and > nothing obviously weird or bad happened. Bor-ing! Maybe try fewer unit > tests next time, to add more excitement to the upgrade process? It also installed effortlessly and worked perfectly on Gentoo x86 and Ubuntu amd64. The only difference I actually noticed from 8.2 was the lack of deprecation warnings from Python 2.6. (Skip to the bottom if you don't care about the detailed pain of installing Twisted from source on Windows.) Then I tried to install it on Windows XP. "python setup.py install" whines about a missing vcvars.bat. I think that means I need to install a C compiler. I download Visual C++ 2008 Express Edition with Service Pack 1 from microsoft.com. Huge license agreement, tiny window, making it almost impossible to actually read it all. Fearing all that legalese, I abort the install and fetch my seven-year-old daughter and ask her to install it, thus preserving my right to someday publish benchmarks of SQL Server without Microsoft's permission. (Yes, it would be cruel to preserve my own rights while sacrificing hers, but she's too young to sign legally binding contracts.) Then we wait forever for the world's slowest installer. (Express Edition. Heh.) It finally finishes and then tells her to reboot. I thank her and send her back to playing. I rerun Twisted's "python setup.py install" I see some errors about missing epoll.h, but the install continues. I finally start up my game server. "To Help Protect My Computer, Windows Firewall has blocked some features of this program." (Funny, it doesn't pop up that same warning if someone tries to use IE or Outlook.) I unblock python. I see deprecation warnings for md5, that I thought were fixed in Twisted 9. So maybe I'm still running 8.2. I start python, import twisted, and print twisted.version. 9.0.0pre1. But I might have a mixed version install, so I go to Add/Remove Programs and uninstall Twisted 8.2, then run "python setup.py install" again to reinstall Twisted 9. I start my game server again, and get "No module named zope.interface" I guess the Twisted 8.2 binary installer installed it for me and when I uninstalled it I lost it, so now I need to install it myself. I fetch a zope interface tarball, gunzip it, untar it, and run "python setup.py install" on it. I start my game server again, and get "ImportError: No module named pkg_resources." I go back to the Twisted directory and redo "python setup.py install" Same error. So I guess the zope interface install failed in some odd way without warning. I go back to the zope interface page, and see a note that says to just use the .exe installer on Windows. But there isn't one. I hunt around the zope site for several minutes and fail to find one. (There's one for all of zope, but that seems like overkill.) I go to PyPI and find a zope interface egg for Windows. I double-click on the egg and Windows has no idea what it is. I go to PyPI and find a setuptools .exe installer, install that, then use setuptools' ez_setup.py to install the Zope Interface egg. And then Twisted 9.0.0pre1 works fine. So the moral of the story is that installing software from source on Windows really is way too hard, and Windows users really do need binary installers. But Twisted 9 did eventually work for me on Windows. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] zope.interface
On 2009.12.14 11:48:48 +, Phil Mayers wrote: > Jared Gisin wrote: > > Is there a good document that explains why zope.interface is necessary > > to the twisted project? > > A counter-question: Is there a good document that explains why people > care? It's just another dependency isn't it? Sure. But Twisted has a very small number of dependencies (I think Python and Zope Interface are the only ones that are absolutely required) so it does increase the total effort to install Twisted on an OS without a good package manager by a noticable amount. Zope Interface is a good library, and I think the Twisted team made the right call when they decided to use it rather than continue to maintain their own interface library. It's not their fault that installing Python dependencies on some operating systems is still way harder than it should be. It would be great if general Python installation and packaging tools improved to the point where one could easily install Twisted and all its dependencies on any supported OS using PyPI, as easily as we can on Linux or BSD using the OS package manager, or as easily as we can install Perl packages from CPAN, or as easily as we can run Java stuff using Java Web Start, etc. Enough smart people care about this problem that I'm pretty sure it'll be solved over the next couple of years. Which is too far away to save me from having to write big nasty custom Windows and MacOS installers for my current game, but might mean that this is the last time I ever have to do it. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] order of remote requests
On 2009.12.29 12:53:07 +0100, Wolfgang Rohdewald wrote: > On Tuesday 29 December 2009, Andrew Bennetts wrote: > > Wolfgang Rohdewald wrote: > > > when I send two remote calls A and B over the network, in that > > > order, using the twisted perspective broker callRemote method, > > > is it granted that A is processed first on the other side even > > > if B arrives first? Or should I only send B after I got an > > > answer to A? > > > > It depends on the server, and what exactly you want to happen. > > > > Perspective broker can have many concurrent remote calls on the > > same connection, and depending on how the server is implemented it > > might process those sequentially in the order they arrive, or > > concurrently, or even in an arbitrary order. > > > > So if call A really must be completed before starting B, then the > > client must not make call B until it knows A has finished. > > I should have given more details, sorry. > > I am writing both the server and the client with twisted pb. > The server (a game server) sends messages to four player clients. > > Say the game server sends messages A and B to a player client. > > Is it granted that the player client always processes A first even > if the internet routes the messages differently and B arrives > first? TCP guarantees in-order message delivery. If you send A before B over the same TCP connection, and A and B reach the client, then the client will receive A before B. Each Twisted process is single-threaded by default. So if you use TCP, and keep the message processing code simple, then I think you'll get messages processed in the order you expect. I've never noticed a problem with out-of-order messages in my PB game, anyway. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Using Twisted and GTK2
On 2010.02.03 13:48:39 +0100, Ramiro Alba Queipo wrote: > I recently new about Twisted and saw it as the best way to deal with a > client/server network application (I use python long time ago). Really > wonderful. I now intend to write a GTK client that can send commands and > transfers files to the server and from it. > I started to learn how to write applications not using 'Perspective > Broker', but when I reached to the example pbgtk2.py to authenticate and > send echo commands, I could no see a way the same with a PB. Is that > true?. Is, anyway the best alternative (I mean PB) to write the > application I intend to? PB is fine. AMP or Foolscap might be better for some tasks, but PB is mature and works. PB features integration with Cred, which is Twisted's pluggable authentication mechanism. http://twistedmatrix.com/documents/current/core/howto/cred.html Cred is very flexible. The integration between PB and Cred isn't quite as flexible, but it works fine for straightforward cases. If you're looking for larger sample code, I have a game that uses PyGTK and PB and Cred, here: http://github.com/dripton/Slugathon -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Using Twisted and GTK2
On 2010.02.04 08:23:20 -0500, Alexandre Quessy wrote: > Though, as far as I know, PB is not transfered in clear text. That > means it's a little harder to debug the TCP streams, using a tool like > Wireshark. Am I wrong? The PB wire format is documented here: http://twistedmatrix.com/documents/9.0.0/api/twisted.spread.jelly.html It's human-readable S-expressions. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] How to use AmpList?
On 2010.03.27 23:45:42 +0200, Vlad Shevchenko wrote:
> Hi,
> can't find any reference of how to use AmpList. What should be passed to
> callRemote? In command: arguments = Amp.List(...) or arguments = [('param',
> AmpList(...))]? Responder params?
>
> Please, advice some refs or examples.
There are a few small examples in twisted/test/test_amp.py
test_ampListCommand might be a good starting point.
--
David Riptondrip...@ripton.net
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Question about processes in python
On 2010.04.12 16:23:54 +0200, Gabriel Rossetti wrote: > Jason J. W. Williams wrote: > > You might want to try Python's multiprocessing code, it works very > > well. I've had pretty good luck using it with Twisted inside the > > processes too. The multiprocessing library handles the work of setting > > up the send and receive file descriptors. > yes, I was going to use that but I read in several threads that this > didn't work correctly in Twisted IMO Jason probably has a lurking bug that he just hasn't noticed so far. Same thing with using subprocess instead of the Twisted equivalent inside a Twisted reactor. IMX it works 99% of the time. Murphy's Law says it'll pass cursory testing and then fail in production. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Question about processes in python
On 2010.04.12 09:39:21 -0600, Jason J. W. Williams wrote: > Haven't had any issues yet. Twisted imports occur inside the process > function. The app was originally written as a purely blocking > multiprocessing app and rewritten to use Twisted inside the > sub-processes. It's passed all automated and hand tests without an > issue. Is there a reason importing Twisted inside sub-process should > not work? Here's JP's canonical answer: http://stackoverflow.com/questions/1948641/twisted-threading-with-subprocess-popen I've seen this problem in real code. We had a PyGTK + Twisted program that erroneously used subprocess in one place. 2% of the time, it caused an exception. 98% of the time, it worked fine. Classic race condition. Could be you have a similar bug but it never actually manifests on your combination of code, OS, and hardware. Hard to say. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] suggestion for a file transfer protocol
On 2010.05.11 12:20:27 +0200, Gabriele Lanaro wrote: > I'm trying to develop a simple application that let communicate two > computers in a LAN for transferring files. My idea is that: > > using dbus/zeroconf each machine can see other's service and can connect > with it (I need something without authentication or so) It's a good concept, as long as all machines are on the same subnet and they all have a zeroconf client installed. I've had very bad experience with the old version of dbus-python that ships with RHEL 5. It just doesn't work reliably. But if you're on a more modern distrobution with newer versions of everything, maybe it'll all work great. > having a lan with 2 machines > > A asks B if he can send a file > > if B accepts, begin transferring > if B refuse, send an error back > > It would be nice to transfer big files (not loaded all in memory) and the > ability to stop/resume the upload/download and the integrity check. > > Is there a protocol that let me do this stuff or is better to implement a > protocol on my own? (I'd like that the system is flexible to further extend > for example, implement shared folders and so on) If a pre-built program like scp or rsync doesn't do exactly what you want, and you want the ability to extend it later, then I think it's reasonable to do your own protocol. > In the latter case it's convenient to use one port or two ports like ftp > does? I prefer to only use one port if possible, because it's simpler. And it simplifies dealing with firewalls, if you ever have to do that. As long as you split large files into reasonably small chunks so the data packets don't cause excessive latency for the control packets, it should work fine. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Don't Thread On Me t-shirt proposal
On 10/20/10 17:10, Laurens Van Houtven wrote: > Feedback is welcome: do you think the layout or relative sizes should be > changed? Is the apostrophe being missing okay? I think the missing apostrophe looks bad. Otherwise, I like the shirt. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] TextEntry
On 11/19/10 09:25, Glyph Lefkowitz wrote: > (In my personal opinion, modal dialogs are terrible UI anyway, and > should be avoided. You should be able to prompt the user without > interrupting everything else they're doing in the program.) I agree for 99% of dialogs, but modal dialogs actually make sense for a few things. For example, the "Do you really want to quit the program without saving?" dialog in many editors. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Removing Python 2.6 Support after Twisted 15.1
On 03/19/2015 08:05 AM, Phil Mayers wrote: > On 18/03/15 23:57, Glyph Lefkowitz wrote: > >> Rather than just suggest we preserve the status quo and stay on 2.6 >> forever to do indefinite free work to support Red Hat's obsolescence > > You *definitely* shouldn't do that. Push back on RedHat, and tell > customers to push back on RedHat. I say this to other projects. Red Hat supports old versions of RHEL forever because they have customers that want that (because they have legacy apps that they want to keep working with minimal maintenance, but who still want security patches), and are willing to pay for it. It's a good thing that someone provides that option, even if we don't all need it. That doesn't mean Twisted needs to support old versions of RHEL forever. RHEL 7 has been out for a while, and comes with Python 2.7 as the default. So even if Twisted drops support for Python 2.6, it'll still work on RHEL. (Just not on old versions of RHEL.) Furthermore, SCL makes Python 2.7 available (in /opt) on RHEL 5 and 6 and CentOS 6, without users needing to build it themselves. So even those who are stuck with old versions can install SCL, then use pip to install new Twisted. IOW, I wouldn't worry about it. Drop support for Python 2.6 when it makes sense for Twisted. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] AMP with a long-lived connection - results and deferred?
On 10/28/2015 02:24 AM, Oon-Ee Ng wrote:
> Working off the example ampserver.py and ampclient.py examples, I
> wanted to build a client which maintains a single connection while
> allowing the passing of messages back and forth. However I'm stuck at
> one of the most basic steps, getting back the result (without
> 'completing' the connection).
>
> The ampclient.py example simply connects, does a callRemote on the
> resulting protocol, and adds callbacks to extract the result from a
> dictionary and print it. The deferred being used in this case is
> produced by connectProtocol.
>
> I'm trying to write a client (inheriting from amp.AMP) which is
> embedded in a kivy GUI, something which is quite possible using the
> _threadedselect reactor.
>
> On connection, I use the connectionMade method to save the 'self' (in
> this case, the client inheriting from amp.AMP) in my kivy app. I can
> then call a function which does a callRemote on this saved client,
> which indeed triggers the server appropriately.
>
> The callRemote returns a deferred (from reading docs online, a remote
> reference). I can't figure out what to do with it, specifically in
> terms of getting the result ('total', when calling Sum from
> ampserver.py).
>
> Assistance much appreciated.
Basically, callRemote returns a deferred, that you can add callbacks and
errbacks to, which will be called when the remote call succeeds or
fails. On success, the callback will receive an argument equal to the
return value of the remote callable that you called. On failure, the
errback will receive an error argument.
I have an old GUI (PyGTK, not Kivy) chat over AMP example at
https://github.com/dripton/ampchat that might help you. (Though I think
all of my commands return boring responses.)
--
David Riptondrip...@ripton.net
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] SURVEY: Have you submitted a patch to Twisted and it never got in?
On 07/01/11 12:08, Itamar Turner-Trauring wrote: > In order to have at least some anecdotal evidence -- > > If you've submitted a patch to Twisted (or started a branch) and it never > made it in, how did that happen? I imagine reasons might include a review > request to write tests, redesign requests, getting distracted, "it works > for me", design discussions that never got anywhere... What happened in > your case? I made it through the first several hurdles (working code, following coding standards, unit tests for everything) but then hit a legitimate reverse compatibility concern that kept my patches from landing. Someone eventually came up with a good solution, but the time gap meant that other things had changed and/or were about to change in Twisted, and gave more people a chance to bikeshed, which gave me less confidence that whatever I eventually finished would land. So I punted and waited for someone with more political clout to take over. Working with patches because you don't have svn commit rights is annoying, but this annoyance is a relatively minor fixed cost. The real issue, for controversial features, is achieving consensus, and then getting your feature in before consensus is lost. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Problem passing objects with perspective broker
On 01/29/12 00:57, Fourat ZOUARI wrote:
> Whenever i try to pass an object (aClass) with a property with a second
> object (bClass) Am getting:
>
> *Unpersistable('Unpersistable data: instance of class amodule.bClass
> deemed insecure')*
>
>
> The aClass is copyable and unjellied, the bClass is not.
>
> Here's my code:
>
> server.py:
>
> from twisted.application import internet, service
> from twisted.spread import pb
> from amodule import PBServer
>
> application = service.Application("Test app")
>
> # Prepare managers
> clientManager = internet.TCPServer(8282,
> pb.PBServerFactory(PBServer()));
> clientManager.setServiceParent(application)
>
> if __name__ == '__main__':
> print "Run with twistd"
> import sys
> sys.exit(1)
>
>
> amodule.py:
>
> from twisted.spread import pb
>
> class bClass:
> """This is not a Copyable class
> """
>
> class aClass(pb.RemoteCopy, pb.Copyable):
> b = bClass()
> pb.setUnjellyableForClass(aClass, aClass)
>
> class PBServer(pb.Root):
> def remote_echo(self, a):
> return a.b
>
> Is there anyway to unjelly aClass and all the objects it can contain ?
> because unjellying included object is a headache and may turn into dirty
> code ...
If you make bClass inherit from pb.Copyable and do
pb.setUnjellyableForClass(bClass, bClass) then everything should work.
Twisted knows how to do what you want, but it's refusing to do because
you haven't marked bClass as safe to pass over the wire.
--
David Riptondrip...@ripton.net
___
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Twisted 12.0.0 released
On 02/11/12 08:35, Thomas Hervé wrote: > On behalf of Twisted Matrix Laboratories, I am honored to announce the > release of Twisted 12.0. Woot! Thanks everyone. > Twisted no longer supports Python 2.4, the latest supported version is 2.5. s/latest/earliest/ 12.0.0.pre1 has been rock solid for me for a couple of weeks, and now 12.0.0 has been equally stable for me for a couple of minutes. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Clean pb solution for two-way object sync?
On 09/26/2013 02:05 AM, Daniel Sank wrote: I want to implement something functionally equivalent to a network chess game. I first consider how I would do this on a single computer with no network (maybe this is bad thinking). Each piece in the game is represented by an instance of class Agent. Each agent has a .graphics attribute which is an instance of a class from a GUI toolkit library or equivalent. Whenever an agent in the game needs to do something there will be business logic executed by the game objects proper (ie the agents) which will invoke methods on the .graphics objects to update the screen. This sort of structure seems natural as it allows easy integration of drag/drop, mouse click detection etc. It also nicely separates the real business logic from the GUI. I think you have the right idea but that's still a bit too much coupling between the logic and the UI for my taste. I don't want the game logic to have a .graphics attribute; I want the game logic to fling game events to one or more consumers, each of which may or may not be a GUI. (Maybe it's a headless AI player. Maybe it's a logging service. The server shouldn't care.) Now I want to run over the network. The question is how should I set up references between the client and server objects? There's more than one way to do it. Here's my game that uses PB: https://github.com/dripton/Slugathon I used PB (because AMP and Foolscap didn't exist yet), but I didn't use the fancy bits of PB like Cacheable, because I strongly prefer simple remote method calls to fancy remote objects. But if you grep for callRemote, remote_, and perspective_, you can see how I did it. As noted above, my game server flings events (see Action.py for what they look like) to both GUI and AI clients. The actions are just little value objects that happen to inherit from pb.Copyable and pb.RemoteCopy for convenience, though they just as easily be JSON blobs. Of course, it's probably much easier to just use Cacheable. It comes down to programmer preference. One piece of advice: do the network code first and always exercise it, even when playing on a single computer. Every time I've written a single-machine game first then tried to add networking later, the networking has been a mess to debug. -- David Riptondrip...@ripton.net ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
