Re: How to write an API for a Python application?
In article <[EMAIL PROTECTED]>, Cameron Laird <[EMAIL PROTECTED]> wrote: [...] >Indeed, Paul: those who scorn CORBA often *do* re-create it, >poorly. > >I still don't think it's the right answer for Mr. Kshepitzki. >Pyro might be perfect. My own instinct is to start even more >primitively, with a minimal asynchat client and server. I've >looked through the *Cookbook*, and see that it doesn't have >what I want. Maybe it's time Phaseit donate one of the >little models we use ... Pyro is definitely of no use to the original poster. He specifically wants to access his Python server from languages other than Python. To me, the situation sounds complex enough, especially with the need for callbacks, that CORBA is an ideal solution. At the expense of a small amount of boilerplate code, all the communication issues are handled for you. In this day and age, why would you want to write code that deals with sockets apart from the most specialist situations? Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Regular expressions and the global interpreter lock
Hi, I have encountered a problem with the re module. I have a multi-threaded program that does lots of regular expression searching, with some relatively complex regular expressions. Occasionally, events can conspire to mean that the re search takes minutes. That's bad enough in and of itself, but the real problem is that the re engine does not release the interpreter lock while it is running. All the other threads are therefore blocked for the entire time it takes to do the regular expression search. Is there any fundamental reason why the re module cannot release the interpreter lock, for at least some of the time it is running? The ideal situation for me would be if it could do most of its work with the lock released, since the software is running on a multi processor machine that could productively do other work while the re is being processed. Failing that, could it at least periodically release the lock to give other threads a chance to run? A quick look at the code in _sre.c suggests that for most of the time, no Python objects are being manipulated, so the interpreter lock could be released. Has anyone tried to do that? Thanks, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: How to write an API for a Python application?
In article <[EMAIL PROTECTED]>, Piet van Oostrum <[EMAIL PROTECTED]> wrote: >A more lightweight solution might be Ice. <http://www.zeroc.com/ice.html> >It is architecturally similar to Corba, but with less overhead. More lightweight and less overhead in what sense? The performance measurements I've seen show that Ice is significantly slower than many CORBA implementations. If you mean more lightweight in terms of ease of use, I don't see how. I quite agree that it's more lightweight in terms of size of specification and difficulty for the ORB implementor, but that's largely irrelevant to the users of the technology. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: How to write an API for a Python application?
In article <[EMAIL PROTECTED]>, Piet van Oostrum <[EMAIL PROTECTED]> wrote: >On http://www.zeroc.com/performance/ they compare it with TAO and it seems >to be faster. It looks also a bit simpler. I don't have experience with Ice >myself but a colleague of mine experimented with it and was enthousiastic. > >It could well be that it is comparable in speed to omniORB, which is my >favorite platform for Corba on Python, or maybe even slower. Do you have >benchmark results about that? I've not done any benchmarks for Python (yet), but I've just posted some results to comp.object.corba that show omniORB is a lot faster than Ice for many things. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for CORBA (omniORB) examples
In article <[EMAIL PROTECTED]>, <[EMAIL PROTECTED]> wrote: >can we script CORBA objects/events with python. my search for simple >examples has not yielded any results. any help appreciated. There are some simple examples in the omniORBpy distribution. There are a few more examples in this presentation I gave: http://www.grisby.org/presentations/py10code.html Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Deadlock detection
Hi, Does anyone know of a deadlock detector for Python? I don't think it would be too hard to hook into the threading module and instrument mutexes so they can be tested for deadlocks. I've googled around but I haven't found anything. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Deadlock detection
In article <[EMAIL PROTECTED]>, Josiah Carlson <[EMAIL PROTECTED]> wrote: > >Duncan Grisby <[EMAIL PROTECTED]> wrote: >> Does anyone know of a deadlock detector for Python? I don't think it >> would be too hard to hook into the threading module and instrument >> mutexes so they can be tested for deadlocks. I've googled around but I >> haven't found anything. > >I'm not aware of a deadlock dector for any language. I propose that a >completely working deadlock detector is NP-Complete, as it would, I >believe, necessitate solving the halting problem. As Paul Du Bois has pointed out, NP complete is much easier than the halting problem (i.e. the halting problem simply can't be done in general, no matter how much time you have). Aside from that, you're right that statically analysing code for deadlocks is equivalent to the halting problem, and therefore impossible. What I'm looking for, though, is a dynamic detector that detects deadlocks at run-time when they happen. Doing that is well understood, and there are plenty of systems that do it. I just haven't been able to find one for Python. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Ice 2.0 released
In article <[EMAIL PROTECTED]>,
Michi Henning <[EMAIL PROTECTED]> wrote:
[...]
>Instead of compiling the definition, you can write:
>
>Ice.loadSlice("Color.ice")
>import M
>
>print "My favourite color is ", M.Color.blue
Just like this then?
omniORB.importIDL("Color.idl")
import M
print "My favourite color is ", M.blue
Cheers,
Duncan.
--
-- Duncan Grisby --
-- [EMAIL PROTECTED] --
-- http://www.grisby.org --
--
http://mail.python.org/mailman/listinfo/python-list
Re: logging in omniORB for python
In article <[EMAIL PROTECTED]>, Diez B. Roggisch <[EMAIL PROTECTED]> wrote: >> Does anyone know, were the log data is? And how I can store it in a file? >> By the way I ask also the omniORB mailing list, but here are the python >> experts. > >it gets written to stdout - and then looks like this: To avoid any confusion, it actually gets written to stderr, not stdout. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: omniorbpy: problems sending float values
In article <[EMAIL PROTECTED]>, Juergen <[EMAIL PROTECTED]> wrote: >I've got a problem sending floating point values to an corba server. >With other datatyes like short or string it works fine. Upgrade to the latest stable release, omniORB 4.0.7 and omniORBpy 2.7. The problem you are seeing is due to a conflict between some marshalling code in omniORB 4.0.6 and your compiler's use of strict aliasing. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Components for a client/server architecture
In article <[EMAIL PROTECTED]>, Samuel <[EMAIL PROTECTED]> wrote: [...] >> Sounds like CORBA to me. CORBA has a very mature and good implementation >> for Python called OmniORB, and interoperability with other orbs (the >> ones available for e.g. Java) is very good - as CORBA as standard is >> mature. > >I have worked with Bonobo (an implementation of CORBA) before, though not >on the network - it is fairly complex. But I did not think of using it >for this purpose, it might actually make sense. I'll have to look into >the transport protocol more. To be clear, Bonobo is not an implementation of CORBA. It is a (somewhat byzantine) implementation of a component model on top of CORBA. CORBA certainly has some complex corners, but for many of the most common application requirements it is pretty simple, and buys you a lot in terms of cross-platform compatibility and clarity of interfaces. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple omniORBpy example throws exception with error 0x41540002
In article <[EMAIL PROTECTED]>, Samuel <[EMAIL PROTECTED]> wrote: [...] >Ah, I see now how this works. I happen to run Ubuntu here, so I tried >the following: > >- sudo apt-get install orbit-name-server-2 >- orbit-name-server-2 & >- Add to /etc/omniORB4.cfg: >InitRef = NameService=IOR:01002b00... >(where everything starting from "IOR:" is the output given by orbit- >name-server-2. > >However, this does not seem to change the behavior. Any hints? I think ORBit is configured to only listen on its proprietary Unix domain socket protocol by default, not TCP, so omniORB doesn't know how to talk to it. You should either tell ORBit to listen on TCP (Google for how), or use omniORB's naming service, which listens on TCP by default. Alternatively, you don't particularly need to use a naming service if you don't want to. You can modify the example to output an IOR string for the object reference rather than trying to register it in the naming service. Print the result of orb.object_to_string(adderObjref) rather than the naming service stuff. The example you are looking at was given as a tutorial with me speaking, so it's a bit light on details of what's going on. You might find the introduction in the omniORBpy manual more useful: http://omniorb.sourceforge.net/omnipy3/omniORBpy/omniORBpy002.html or chapter 2 in the PDF version: http://omniorb.sourceforge.net/omnipy3/omniORBpy.pdf Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
ANNOUNCE: omniORB 4.1.1 and omniORBpy 3.1
I am pleased to announce that omniORB 4.1.1 and omniORBpy 3.1 are now available. omniORB is a robust, high performance CORBA implementation for C++; omniORBpy is a version for Python. You can download them in source and Windows binary forms from SourceForge: http://sourceforge.net/project/showfiles.php?group_id=51138&package_id=44914&release_id=544978 http://sourceforge.net/project/showfiles.php?group_id=51138&package_id=48639&release_id=544980 These are mainly bug fix releases, with a number of minor new features. See the release notes and bug lists for more details: For omniORB 4.1.1: http://sourceforge.net/project/shownotes.php?release_id=544978&group_id=51138 http://omniorb.sourceforge.net/bugs/bugfixes-410.html For omniORBpy 3.1: http://sourceforge.net/project/shownotes.php?release_id=544980&group_id=51138 http://omniorb.sourceforge.net/pybugs/bugfixes-30.html Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Approaches of interprocess communication
In article <[EMAIL PROTECTED]>, exhuma.twn <[EMAIL PROTECTED]> wrote: >Supposing you have two separate processes running on the same box, >what approach would you suggest to communicate between those two >processes. [...] >* Webservices > Advantage: Relatively easy to use, can work across different >languages > Disadvantage: Even more overhead on the TCP/IP side that simple >sockets, as really bulky SOAP messages need to be passed around. > >* CORBA -- similar to webservices but more complicated to code. Lots of people say that, but I don't think it's true. Obviously as the maintainer of a CORBA implementation I'm biased, but take a look at some examples of code that implement SOAP clients and servers and compare it to similar CORBA code. Especially in Python, the SOAP code tends to be incredibly verbose and complex, and the CORBA code really small and simple. My recommendation would be that for simple communications where performance isn't important use XML-RPC; for more complex cases where performance is a bit more important but you don't need anything except Python, use Pyro; for cases where performance is particularly important and/or you need cross-language communications, use CORBA. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- Posted via NewsDemon.com - Premium Uncensored Newsgroup Service --->>>>>>http://www.NewsDem -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0 - is this true?
In article <[EMAIL PROTECTED]>,
Terry Reedy <[EMAIL PROTECTED]> wrote:
[...]
>> l = []
>> for line in open("bigfile.txt"):
>> x = random.randint(0,100)
>> if x < 4: l.append(None)
>> else: l.append(line)
>
>So use '' or '\0' instead of None for null lines. Or replace None for
>the sort.
Both '' and '\0' could conceivably be valid data values. The problem
is that the data comes from a dynamically-typed database, so there is
no guarantee what the data values might be. This for loop with a file
is just an example that generates similar-looking data. It bears no
relation to the way the data is actually acquired.
>> And maybe once in a while you end up with something not dissimilar to:
>>
>> l = []
>> for line in open("bigfile.txt"):
>> x = random.randint(0,100)
>> if x < 4: l.append(None)
>> elif x < 5: l.append([line,line])
>> else: l.append(line)
>>
>> In that kind of case it doesn't really matter what happens to list
>> items in the sort order, but it's important it doesn't fail to sort
>> the ones that are strings.
>
>Replace the sublists with a coded string, such as '\0'+line.
Again, this is just an example. As I say, in the real application, the
data has come from a dynamically-typed database. There's no easy way
to coerce all the items to the same type, because the application
doesn't know up-front what the data is going to look like. In some
cases, it may be that all the data items are lists of strings. In
other cases, all or most of the data items will be integers, for
example.
>If sorting is time (and space) critical, you want a straight string sort
>without key *or* cmp function.
That's exactly my point. Currently, the application just builds a list
of values retrieved from the database and asks Python to sort it. No
key or cmp function. With Python 3.0, it's going to require a lot more
work than that.
For my application, Python 3's comparison behaviour is a backwards
step. You can argue all you like that the new behaviour is the "right"
thing to do, and I'd be inclined to agree with you from a
philosophical point of view, but the fact is that it _will_ cause
problems for existing real code. The particularly ironic thing is that
the database's dynamic typing is closely modelled on Python, including
it's ability to gracefully handle mixed-type lists.
Cheers,
Duncan.
--
-- Duncan Grisby --
-- [EMAIL PROTECTED] --
-- http://www.grisby.org --
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0 - is this true?
In article <[EMAIL PROTECTED]>, Aahz <[EMAIL PROTECTED]> wrote: >In article <[EMAIL PROTECTED]>, >Duncan Grisby <[EMAIL PROTECTED]> wrote: [...] >>That's exactly my point. Currently, the application just builds a list >>of values retrieved from the database and asks Python to sort it. No >>key or cmp function. With Python 3.0, it's going to require a lot more >>work than that. > >Unless I misunderstand you, your application is broken already. You misunderstand me. My application is not broken. It handles a wide range of data types, _but they are all sortable_. It doesn't involve any complex numbers. There is no way a complex number could be stored in the database, so that is not an issue. > Complex >numbers have never been sortable. If your application works more like a >SQL database, then any given column will have only one datatype and as >long as you avoid types that cannot be compared against themselves >you're safe. My database isn't like a SQL database. It's dynamically typed, just like Python. The range of types is limited to things that can be marshalled into the underlying storage, so I can absolutely guarantee that they are all sortable in Python 2, even though the permitted range or types is quite wide. [...] >>For my application, Python 3's comparison behaviour is a backwards >>step. You can argue all you like that the new behaviour is the "right" >>thing to do, and I'd be inclined to agree with you from a >>philosophical point of view, but the fact is that it _will_ cause >>problems for existing real code. The particularly ironic thing is that >>the database's dynamic typing is closely modelled on Python, including >>it's ability to gracefully handle mixed-type lists. > >What I think people are arguing about is your use of the word "backward". >I don't think anyone claims that fixing your application will be trivial, >but your application appears to be already broken, and Python 3.0 is >simply forcing you to confront it head-on. The post you've quoted was the first time I used the word "backwards". I didn't start this thread, I merely chipped in when people claimed there were no real applications that would be affected by this change to Python. I do have a real, widely deployed application that will be affected. Claiming that it is not affected, or that it is already "broken" does not change that fact. Several people _have_ in fact claimed that I can't possibly have a hard problem to solve in this respect. In general, the responses can be grouped into those people claiming that it's trivial to fix, and those (like you) claiming that it's ok that it's hard to fix because it's already broken. Neither are true. This issue is not impossible to deal with, of course, it's just one of several things that will mean it's hard for us to migrate to Python 3. What I find disturbing is the attitude that this change to Python's comparison behaviour can't possibly have any downsides and that anyone claiming it does is wrong. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0 - is this true?
In article <[EMAIL PROTECTED]>, Terry Reedy <[EMAIL PROTECTED]> wrote: >Have you written any Python code where you really wanted the old, >unpredictable behavior? I have an object database written in Python. It, like Python, is dynamically typed. It heavily relies on being able to sort lists where some of the members are None. To some extent, it also sorts lists of other mixed types. It will be very hard to migrate this aspect of it to Python 3. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0 - is this true?
In article <[EMAIL PROTECTED]>, Terry Reedy <[EMAIL PROTECTED]> wrote: >> I have an object database written in Python. It, like Python, is >> dynamically typed. It heavily relies on being able to sort lists where >> some of the members are None. To some extent, it also sorts lists of >> other mixed types. It will be very hard to migrate this aspect of it >> to Python 3. > >Very Hard? Several key functions have been suggested on this thread. >Given that 2.x only sorts most but not all types and that the sort is >only guaranteed to be consistent within a session, as I remember, I >suspect you can choose or write something at least as good for your >purposes. Yes, very hard. There are only ever simple types in the lists -- strings, integers, Nones, very occasionally floats, and lists of those things. The sort is always predictable with those types. Just because you can contrive situations to demonstrate unpredictable sorts doesn't mean that all sorts with mixed types are unpredictable. The sorting is in a performance-critical part of the system, so the overhead of evaluating a key function is not insignificant. A key function that returns objects that contrive to emulate the functionality of a comparison function is definitely not appropriate. That area of the system already builds the lists using C++ for speed, so if we ever migrate to Python 3 it will probably be easier to do the whole thing in C++ rather than jump through hoops to make the Python sort work efficiently enough. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0 - is this true?
In article <[EMAIL PROTECTED]>,
Martin v. Löwis <[EMAIL PROTECTED]> wrote:
>> The sorting is in a performance-critical part of the system, so the
>> overhead of evaluating a key function is not insignificant.
>
>Can you easily produce an example? It doesn't have to be real data,
>but should have the structure (typewise) of the real data. I would
>like to perform some measurements. For example, I could imagine that
>
>l = []
>for i in range(1000):
> x = random.randint(0,100)
> if x < 4: l.append(None)
> else: l.append(x)
>
>might adequately model your problem.
Sorry for the delay in replying. Yes, that's not far off. Most of the
time the lists contain strings, though. A better approximation might
be to read lines from a file and randomly replace them with Nones:
l = []
for line in open("bigfile.txt"):
x = random.randint(0,100)
if x < 4: l.append(None)
else: l.append(line)
And maybe once in a while you end up with something not dissimilar to:
l = []
for line in open("bigfile.txt"):
x = random.randint(0,100)
if x < 4: l.append(None)
elif x < 5: l.append([line,line])
else: l.append(line)
In that kind of case it doesn't really matter what happens to list
items in the sort order, but it's important it doesn't fail to sort
the ones that are strings.
Cheers,
Duncan.
--
-- Duncan Grisby --
-- [EMAIL PROTECTED] --
-- http://www.grisby.org --
--
http://mail.python.org/mailman/listinfo/python-list
Re: glibc detected *** python: corrupted double-linked list
In article , David Cournapeau wrote: [...] >It is very unlikely the problem is in glibc - I would check your code >carefully first :) On Linux, the following are useful: You are right that it is extremely unlikely that the bug is in glibc. However, it is not impossible. Several of my colleagues and I spent months trying to track down a bug where Python would occasionally crash with a segfault while iterating through large lists. Of course the problem only ever happened on customer sites, and not in our lab. I eventually tracked it down to a bug in glibc's realloc() implementation, where once in a blue moon it would fail to copy all the data into the newly-allocated buffer. It turned out to be a bug that had been found before, but had been closed as invalid a few months earlier: http://sources.redhat.com/bugzilla/show_bug.cgi?id=5743 The fix had then been applied about a week before I found it myself. I'm not sure that particular fix has found it into any of the major Linux distributions yet. Cheers, Duncan. -- -- Duncan Grisby -- -- dun...@grisby.org -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
Re: Does omniORBpy 3.2 supports DII?
In article <[EMAIL PROTECTED]>, Ilan <[EMAIL PROTECTED]> wrote: >My apologies if this is not the correct forum for thses quiestions, by >I'm rather new to python+CORBA > >Has anyone used omniORBpy as CORBA implementation? are there any >drawbacks? As omniORBpy's author, I'm biased, so I'll let other people answer that... >Does omniORBpy 3.2 supports the Dynamic Invocation Interface? No, it doesn't. You can use Python's normal dynamic features to build dynamic requests, though. Cheers, Duncan. -- -- Duncan Grisby -- -- [EMAIL PROTECTED] -- -- http://www.grisby.org -- -- http://mail.python.org/mailman/listinfo/python-list
