Compiling python without ssl?
Is there any way to compile python (3.1.3, in case it matters) without ssl support? OpenSSL is on my system, and configure finds it, but I can't find a way to tell configure to explicitly ignore it. I need a version of python without ssl for trade compliance reasons (I don't make the dumb rules, I just follow 'em), and we used to be able to just remove the ssl module after the build. With more recent releases, though, this approach causes problems with e.g. hashlib. Ideally I'd like something like "configure --disable-ssl", but I can also do surgery on configure or something if that's what it takes. Thanks in advance! Austin -- http://mail.python.org/mailman/listinfo/python-list
Python users in Stavanger, Norway?
Hei! I'm a Python developer in Stavanger, Norway looking for other Python users/developers/etc. who might be interested in starting a local user group. Anyone interested? This group might actually evolve into a general programming/computer group, depending on the mix of people, but I think that's probably a good thing. I know there are a lot of computer types in the area, but there doesn't seem to be much of a "community". I'd like to change that if we can, so let me know if you're interested. Austin -- http://mail.python.org/mailman/listinfo/python-list
Some C-API functions clear the error indicator?
I've noticed that several (many?) python functions seem to clear the
error/exception indicators when they're called from a C/C++ program.
For example, both PyImport_ImportModule and traceback.extract_tb()
(called via the function call methods) do this: if error indicators
are set prior to their call (as indicated by PyErr_Fetch, and
including a call to PyErr_Restore), I see that they are unset (using
the same method) after the call. This happens even when the functions
succeed.
The functions that do this don't seem to indicate in their
documentation that this will happen. So first, does anyone know why
this is happening? Is it because of the context in which I'm making
the calls? Is there any pattern or reason behind which functions will
do this? Or am I just doing something wrong?
If the problem is context-dependent (e.g. I haven't properly
established a call stack, or something of that flavor), any pointers
on doing things properly would be great.
Here's some example code demonstrating the problem:
---
#include
int main(int argc, char** argv)
{
Py_Initialize();
// Cause an IndexError
PyObject* list = PyList_New(0);
PyObject* obj = PyList_GetItem(list, 100);
PyObject *t = NULL, *v = NULL, *tb = NULL;
// Verify that we see the error
PyErr_Fetch(&t, &v, &tb);
assert(t);
PyErr_Restore(t, v, tb);
// Import a module, which seems to be clearing the error indicator
PyObject* mod = PyImport_ImportModule("sys");
assert(PyObject_HasAttrString(mod, "path"));
// Verify that the error indicator has been cleared
PyErr_Fetch(&t, &v, &tb);
assert(!t); // <=== The error is gone!
PyErr_Restore(t, v, tb);
Py_Finalize();
return 0;
}
---
Thanks in advance.
Austin
--
http://mail.python.org/mailman/listinfo/python-list
Re: Some C-API functions clear the error indicator?
Maybe I'm not following what you're saying. In my case, I already know that an exception has been thrown. In the course of processing that exception, I call another function which, for whatever reason and even when it succeeds, clears the exception indicators. How can I address this issue by checking function calls for failure? Austin On Fri, Jan 29, 2010 at 9:04 PM, Gabriel Genellina wrote: > En Fri, 29 Jan 2010 11:37:09 -0300, Austin Bingham > escribió: > >> I've noticed that several (many?) python functions seem to clear the >> error/exception indicators when they're called from a C/C++ program. >> For example, both PyImport_ImportModule and traceback.extract_tb() >> (called via the function call methods) do this: if error indicators >> are set prior to their call (as indicated by PyErr_Fetch, and >> including a call to PyErr_Restore), I see that they are unset (using >> the same method) after the call. This happens even when the functions >> succeed. > > It's simple: you have to check *every* function call for failure. Many > functions return new object references and you have to properly decrement > them in case of failure, so in most cases this means that you have to check > each and every call. > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: Some C-API functions clear the error indicator?
The original post was, in a nutshell, the "use case"; it's very scaled down the from the real example, and not intended to "make sense". The notion on which I was (apparently incorrectly) basing my exception translation was that I could 1) get and reset references to the error indicators, 2) call other python methods that don't themselves throw, and 3) later retrieve the same "active" exceptions. I was relying on this ability to "re-fetch" exceptions insofar as the functions in my code which dealt with python exceptions all looked up the exception separately. The predicate that "a successful function won't modify the error indicators" appears to be wrong, however, and I've modified my code accordingly. Austin On Sat, Jan 30, 2010 at 1:11 AM, Gabriel Genellina wrote: > En Fri, 29 Jan 2010 18:25:14 -0300, Austin Bingham > escribió: > >> Maybe I'm not following what you're saying. In my case, I already know >> that an exception has been thrown. In the course of processing that >> exception, I call another function which, for whatever reason and even >> when it succeeds, clears the exception indicators. How can I address >> this issue by checking function calls for failure? > > Maybe if you provide an actual use case we can suggest how to handle it. The > code in your original post does not make any sense to me (except by showing > that PyImport_ImportModule does clear the error indicator). If you already > know there was an error, and you even have retrieved the error details, why > do you care if the error indicator gets reset? > > -- > Gabriel Genellina > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: Some C-API functions clear the error indicator?
That makes a lot of sense. And if I take the approach that any Py* function might do this, it actually looks like I can simplify my code (rather than managing some list of ill-behaved functions or something.) Thanks! On Fri, Jan 29, 2010 at 3:58 PM, Duncan Booth wrote: > Austin Bingham wrote: > >> The functions that do this don't seem to indicate in their >> documentation that this will happen. So first, does anyone know why >> this is happening? Is it because of the context in which I'm making >> the calls? Is there any pattern or reason behind which functions will >> do this? Or am I just doing something wrong? >> > (Just guessing here) > I would expect that any function that executes Python code will clear the > error. > > I think that has to happen otherwise the Python code will throw an > exception whenever it gets round to checking for errors. In the past I've > found that if you fail to check for an error in C code before returning to > the interpreter you get the exception thrown a few instructions later, so > something similar would probably happen if you call other Python code from > C. > > If it is anything that executes Python then that would include any function > that creates or destroys an object with Python constructor or destructor > code. or that compares or otherwise operates on instances defined in > Python. In particular it might mean that any function that doesn't appear > to clear the error could do so in a slightly different situation. > > -- > Duncan Booth http://kupuguy.blogspot.com > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Walking an object/reference graph
I'm looking for the proper way to "walk" a graph of python objects. My specific task is to find all objects of a given type that are referred to (transitively) by some starting object. My approach has been to loop through the object's attributes, examining each, and then recursing on them. This approach has immediate problems in that I immediately hit infinite recursion due to method-wrappers; I'm not sure *exactly* what's going on, but it appears that method-wrappers refer to method-wrappers ad infinitum. If I add code to avoid this issue, others soon crop up, so before I wrote too much more code, I thought I'd check with experts to see if there's a better way. So, if I've explained what I'm looking for well enough, does anyone know of a proper way to walk a graph of object references? I need to support cycles in the graph, so keeping track of visited object IDs or something will have to be employed. Also, I need to be able to follow references stored in lists, dictionaries, etc...basically any way to programatically get from one object to another should be followed by the algorithm. I feel like I'm missing something simple, but perhaps this is just a tricky problem for python. Any ideas or insight would be great. Thanks! Austin -- http://mail.python.org/mailman/listinfo/python-list
set using alternative hash function?
If I understand things correctly, the set class uses hash() universally to calculate hash values for its elements. Is there a standard way to have set use a different function? Say I've got a collection of objects with names. I'd like to create a set of these objects where the hashing is done on these names. Using the __hash__ function seems inelegant because it means I have to settle on one type of hashing for these objects all of the time, i.e. I can't create a set of them based on a different uniqueness criteria later. I'd like to create a set instance that uses, say, 'hash(x.name)' rather than 'hash(x)'. Is this possible? Am I just thinking about this problem the wrong way? Admittedly, I'm coming at this from a C++/STL perspective, so perhaps I'm just missing the obvious. Thanks for any help on this. Austin Bingham -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
That's definitely a workable solution, but it still rubs me the wrong way. The uniqueness criteria of a set seems, to me, like a property of the set, whereas the python model forces it onto each set element. Another issue I have with the HashWrapper approach is its space requirements. Logically, what I'm asking to do is switch out a single function reference (i.e. to point at get_name() rather than hash()), but in practice I'm forced to instantiate a new object for each of my set members. On a large set, this could be disastrous. Don't get me wrong...your solution is a good one, but it's just not what I am looking for. Austin On Thu, Oct 15, 2009 at 1:36 PM, Chris Rebert wrote: > On Thu, Oct 15, 2009 at 4:24 AM, Austin Bingham > wrote: >> If I understand things correctly, the set class uses hash() >> universally to calculate hash values for its elements. Is there a >> standard way to have set use a different function? Say I've got a >> collection of objects with names. I'd like to create a set of these >> objects where the hashing is done on these names. Using the __hash__ >> function seems inelegant because it means I have to settle on one type >> of hashing for these objects all of the time, i.e. I can't create a >> set of them based on a different uniqueness criteria later. I'd like >> to create a set instance that uses, say, 'hash(x.name)' rather than >> 'hash(x)'. >> >> Is this possible? Am I just thinking about this problem the wrong way? >> Admittedly, I'm coming at this from a C++/STL perspective, so perhaps >> I'm just missing the obvious. Thanks for any help on this. > > You could use wrapper objects that define an appropriate __hash__(): > > #*completely untested* > class HashWrapper(object): > def __init__(self, obj, criteria): > self._wrapee = obj > self._criteria = criteria > > #override __hash__() / hash() > def __hash__(self): > return hash(self._criteria(self._wrapee)) > > #proxying code > def __getattr__(self, name): > return getattr(self._wrapee, name) > > def __setattr__(self, name, val): > setattr(self._wrapee, name, val) > > #example > def name_of(obj): > return obj.name > > def name_and_serial_num(obj): > return obj.name, obj.serial_number > > no_same_names = set(HashWrapper(obj, name_of) for obj in some_collection) > no_same_name_and_serial = set(HashWrapper(obj, name_and_serial_num) > for obj in some_collection) > > Cheers, > Chris > -- > http://blog.rebertia.com > -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
I guess we see things differently. I think it's quite natural to want a set of unique objects where "unique" is defined as an operation on some subset/conflation/etc. of the attributes of the elements. That's all that the regular set class is, except that it always uses the hash() function to calculate uniqueness. In any event, the notions of set and uniqueness I'm using are well established in other languages, so I don't see why it couldn't be made to work in python. As far as using a dict, that doesn't really solve my problem. It could be part of a solution, I guess, but I would still need functionality extrinsic to the dict. What I want is to make sure that no values in my set have the same name, and dict won't guarantee that for me. A set that calculated uniqueness based on its elements' names, on the other hand, would. Austin On Thu, Oct 15, 2009 at 1:48 PM, Duncan Booth wrote: > It doesn't make sense to use just part of an object as the key for a set. A > set is just a collection of values and there is no key separate from the > value itself. If you build a set from x.name then that works fine, but only > the names are stored. > > What you want in this particular case is a dict: a dict stores both a key > and a value and lets you retrieve the value from the key. > > > -- > Duncan Booth http://kupuguy.blogspot.com > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
set using alternative hash function?
On Thu, Oct 15, 2009 at 2:23 PM, Diez B. Roggisch wrote: > Austin Bingham wrote: > This is a POV, but to to me, the set just deals with a very minimal > protocol - hash-value & equality. Whatever you feed it, it has to cope with > that. It strikes *me* as odd to ask for something else. But I'm not really asking for anything that changes the paradigm of how things currently work. All I need is the ability to say something like this: s = set(hash_func = lambda x: hash(x.name)) If set could be de-hardcoded from using hash(), nothing else about how it works would have to change. Or am I wrong on that? I see that you mention equality in the protocol, but I don't know why a set would need that if a) hash(x) == hash(y) --> x == y and b) hash function must return a 32 bit value (which I think is the case)...so maybe I misunderstand something. I wonder...does the requirement of using hash() have to do with the low-level implementation of set? That might explain the state of things. > The ideal solution would be to have several hash/eq-methods on your object, > and somehow make the surroundings decide which to use. But there is no > OO-pragma for that. But why force objects to know about all of the wacky contexts in which they might be used? Let objects be objects and hash-functions be hash-functions. If someone wants a set where uniqueness is defined by the number of occurrences of 'a' in an object's name, the object should never have to know that...it should just have a name. In any event, it sounds like set doesn't have any notion of switching out its hash function, and that more or less answers my question. Austin -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
On Thu, Oct 15, 2009 at 3:02 PM, Diez B. Roggisch wrote: > Austin Bingham wrote: > You do. Hashes can collide, and then you need equality. Sets are *based* on > equality actually, the hash is just one optimization. ... Right, thanks for clearing that up. Not reading closely enough == public shaming ;) > Because these functions need intimate knowledge of the objects to actually > work. Sure, in python it's easy to define them outside, and just reach into > the object as you wish. But aside this implementation detail, a hash (and > equality of course) always are based upon the object in question. So I > think it's natural to have it defined right there (which __hash__ and > __eq__ show that this seems to be accepted in general). > > And if there were something that would decide on context which of several > implementations to use, you'd have less to worry. As things are, there > isn't such thing (I don't even have the slightest idea what could work), > you are as well off with defining two functions. But this "context decider" you're talking about sounds exactly like what I'm talking about. If I had some class with __hash1__ and __hash2__ (and associated equality operators), you're saying it would be nice to be able to select which to use based on the context (e.g. what type of set I'm using.) It might look like this: s = set(hash_func = lambda x: x.__hash2__, eq_func = lambda x, y: x.__eq2__(y)) And if *that* works for you, do you still have a problem with: s = set(hash_func = lambda x: hash(x.name), eq_func = lambda x,y: x.name == y.name) ? If you don't like those, what would work for you? As far as I can tell, your "context decider" and my "alternative hash functions" are the same thing, and we just need to agree on a syntax. -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
On Thu, Oct 15, 2009 at 3:43 PM, Diez B. Roggisch wrote: > The context-decider isn't the same thing because it isn't designed yet :) > And most probably won't ever be. It's just the abstract idea that > hashing/equality change for one object depending on the circumstances they > are used in, and that one wishes that the decision what to use is as simple > & transparent as possible. Fair enough :) > Your approach is certainly viable, but I guess the current > set-implementation is optimized on working with __hash__ and __eq__ on > objects because for these exist slots in the python object structure in C. > So while you can implement your idea, you will end up passing wrappers as > Chris & me proposed into the current set implementation. Yes, I figured that the guts of set relied on particulars to which we are not privy at the python level. If the syntax let me describe sets the way I've been laying out here, I could probably tolerate the underlying implementation relying on wrappers. > However, it might be worth thinking of proposing this change to the set-type > in general. But then for orthogonality, dicts should have it as well I > guess. Which opens a whole new can of worms. dicts would certainly have to be looked at as well, but I don't think the can would have that many worms in it if we solve the set issue to everyone's satisfaction. In any event, thanks for helping me work through this issue. Austin -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
On Thu, Oct 15, 2009 at 3:50 PM, Mick Krippendorf wrote: > Austin Bingham schrieb: > What you seem to imply is that the hash function imposes some kind of > uniqueness constraint on the set which uses it. That's just not the > case, the uniqueness constraint is always the (in-)equality of objects, > and for this you can override __eq__. But then you must also ensure that > x.__eq__(y) --> y.__eq__(x) & x.__hash() == y.__hash__(). Yes, as was pointed out earlier, I was reading too much into the hash function's role. However, given well behaved hash and equality (which would be a great name for a band), I don't see why those functions need to be defined on the object itself, per se. Right now that's the case because hash() only knows how to call obj.__hash__ (etc. for __eq__). I guess a good analog for what I'm thinking about is list.sort(). It's more than happy to take a comparison operator, and in spirit that's exactly what I'm looking for with sets. > Diez' solution is the pythonic way, and BTW, mathematically and > pythonically sound, wheras, if the hash function would really impose > uniqueness: . . . Yes, my gray c++ roots are showing here; you're right that my brain was secretly expecting the "compiler" to take care of things. Your points about set operations are the strongest in this discussion, and I haven't got a great answer. > Python isn't strong on encapsulation, so "extrinsic" functionality is > quite common. I guess that's part of what's frustrating for me on this issue. Python is generally extremely flexible and open, but in this case I feel like I'm being shut out. Oh well...in the end it looks like there are ways to get what I need, if not what I want. Thanks for the input. Austin -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
On Thu, Oct 15, 2009 at 4:06 PM, Anthony Tolle wrote: > Why not use a dict? The key would be the object name. Pretty much > the same behavior as a set (via the key), and you can still easily > iterate over the objects. To reiterate, dict only gets me part of what I want. Whereas a set with uniqueness defined over 'obj.name' would guarantee no name collisions, dict only sorta helps me keep things straight; it doesn't actually enforce that my values have unique names. Austin -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
On Thu, Oct 15, 2009 at 5:15 PM, Gabriel Genellina
wrote:
> En Thu, 15 Oct 2009 11:42:20 -0300, Austin Bingham
> escribió:
> I think you didn't understand correctly Anthony Tolle's suggestion:
>
> py> class Foo:
> ... def __init__(self, name): self.name = name
> ...
> py> objs = [Foo('Joe'), Foo('Jim'), Foo('Tom'), Foo('Jim')]
> py> objs
I understand Anthony perfectly. Yes, I can construct a dict as you
specify, where all of the keys map to values with name attributes
equal to the key. My point is that dict doesn't really help me enforce
that beyond simply letting me set it up; it doesn't care about the
values at all, just the keys. All that I'm asking for, and I think
it's been made pretty clear, is a set that let's me define a
uniqueness criteria function other than hash(). As has been thrashed
out already, this is not as straightforward as I might have liked.
To put it in code, I want this:
s = set(hash_func = lambda obj: hash(obj.name), eq_func = ...)
...
x.name = 'foo'
y.name = 'foo'
s.add(x)
s.add(y) # no-op because of uniqueness criteria
assert len(s) == 1
Austin
--
http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
On Thu, Oct 15, 2009 at 7:05 PM, Anthony Tolle wrote: > I wrote a quick subclass of set that does something similar, but uses > just one function for the object uniqueness: > > class MySet(set): > def __init__(self, iterable = (), idfunc = lambda x: x): > self.idfunc = idfunc > self.ids = set() > for item in iterable: > self.add(item) > > def add(self, item): > id = self.idfunc(item) > if id not in self.ids: > self.ids.add(id) > set.add(self, item) Yes, what you've got there provides the interface of what I want. And no doubt we could concoct countless ways to implement some version of this. However, if set itself accepted an alternate hash function, then I could do it with no extra overhead. Consider your implementation: it requires an extra set (extra space) and an extra lookup on many operations (extra time.) My original hope was to not have to do that. Austin -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
On Thu, Oct 15, 2009 at 7:49 PM, Ethan Furman wrote: > Austin Bingham wrote: > I'm feeling really dense about now... What am I missing? What you're missing is the entire discussion up to this point. I was looking for a way to use an alternative uniqueness criteria in a set instance without needing to modify my class. > So is that the behavior you're wanting, keeping the first object and > discarding all others? Or is there something else I'm still missing? Yes and yes. I want "normal" set behavior, but I want the set to use user-provided hash and equality tests, i.e. ones that don't necessarily call __hash__ and __eq__ on the candidate elements. Austin -- http://mail.python.org/mailman/listinfo/python-list
Re: set using alternative hash function?
On Thu, Oct 15, 2009 at 8:10 PM, Anthony Tolle wrote: > I think that without a practical example of what this would be used > for, we're all going to be a little lost on this one. > > So far, we've not seen the original problem, only the author's > preferred method for solving it. My guess is there are other, more > pythonic ways to solve the original problem. The original problem was just a question statement: can I use alternative uniqueness functions on a set? Indeed, I proposed an idea, which that was sets could be constructed with user-defined hash and equality functions, the strengths and weaknesses of which have been gone over in some detail. The short answer is that what I was looking for (admittedly, a desire motivated by experiences in other languages) is not really feasible, at least not without a great deal more work. The other proposed solutions all require linearly extra storage, linearly extra time, both, or simply don't solve the problem. And in any event, they miss the point of the original post which was not "How can I get a particular behavior?" (which is fairly trivial, both in my own estimation and as evidenced by the abundance of proposals) but "Can I get a particular behavior in a particular way?" (the answer to which, again, seems to be no.) Austin -- http://mail.python.org/mailman/listinfo/python-list
Re: C/C++ Import
Does the 'python' directory contain a file named '__init__.py'? This
is required to let that directory act as a package (see:
http://docs.python.org/tutorial/modules.html#packages); without it,
you'll see the symptoms you're seeing.
Austin
On Mon, Feb 8, 2010 at 4:56 AM, 7H3LaughingMan wrote:
> To make the background information short, I am trying to take a
> program that uses Python for scripting and recompile it for Linux
> since it originally was built to run on Win32. The program itself was
> designed to be able to be compiled on Linux and someone made there on
> release with source that added python scripting. After some issues I
> got it to compile but now it is unable to import the files that it
> needs.
>
> The program is running the following code...
> PyImport_Import( PyString_FromString("python.PlayerManager") );
>
> This is meant to import the file PlayerManager.py inside of the python
> folder. However it throws the following Python Error (Gotten through
> PyErr_Print())
> ImportError: No module named python.PlayerManager
>
> I am using 2.6.4 so I can't call it by the filename, does anyone know
> how to do a proper import?
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: C/C++ Import
Just to elaborate on Terry's point a bit, sys.path is influenced (in
part) by the PYTHONPATH environment variable. If you find that the
directory containing 'python' is not in sys.path (which you can check
with 'import sys; print sys.path'), add that directory to PYTHONPATH
and try again. This may not be the solution you ultimately end up
using, but it'll get you pointed in the right direction.
Austin
On Mon, Feb 8, 2010 at 5:52 PM, Terry Reedy wrote:
> On 2/7/2010 10:56 PM, 7H3LaughingMan wrote:
>>
>> To make the background information short, I am trying to take a
>> program that uses Python for scripting and recompile it for Linux
>> since it originally was built to run on Win32. The program itself was
>> designed to be able to be compiled on Linux and someone made there on
>> release with source that added python scripting. After some issues I
>> got it to compile but now it is unable to import the files that it
>> needs.
>>
>> The program is running the following code...
>> PyImport_Import( PyString_FromString("python.PlayerManager") );
>>
>> This is meant to import the file PlayerManager.py inside of the python
>> folder. However it throws the following Python Error (Gotten through
>> PyErr_Print())
>> ImportError: No module named python.PlayerManager
>>
>> I am using 2.6.4 so I can't call it by the filename, does anyone know
>> how to do a proper import?
>
> Your 'python' package directory must be in a directory listed in sys.path. I
> would print that check.
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Crypto and export laws
I'm trying to get a handle on how python intersects with crypto-related export control laws in the US and elsewhere. My current understanding, per the PSF's wiki, is that any crypto related and potentially export-sensitive code is in the ssl wrapper, and that, in fact, this only links to the actual encryption implementation (presumably libssl or something.) One caveat is that windows installations may include the ssl implementation. Does this effectively sum up python's exposure to export laws? On a technical level, does removing the ssl module from a distribution remove all references to encryption? Of course I'm not asking for actual legal advice, but can anyone think of any other part of the code that might run afoul of export rules? Thanks. Austin -- http://mail.python.org/mailman/listinfo/python-list
