Re: cPickle fails on manually compiled and executed Python function

2017-07-18 Thread Jan Gosmann
On 07/18/2017 01:07 AM, dieter wrote: "Jan Gosmann" writes: [...] fn = load_pyfile('fn.py')['fn'] [...] "pickle" (and "cpickle") are serializing functions as so called "global"s, i.e. as a module reference together with a name. This means, they cannot handle functions computed in a module (as

Re: cPickle fails on manually compiled and executed Python function

2017-07-17 Thread dieter
"Jan Gosmann" writes: > today I came across some weird behaviour (a bug?) in Python 2.7.13 (on > Linux) with the cPickle module. The pickle module works and so does > the pickle module in Python 3. > > I have a file fn.py with a minimal function definition: > > ``` > def fn(): > pass > ``` >

Re: cPickle - sharing pickled objects between scripts and imports

2012-06-25 Thread Rotwang
On 24/06/2012 00:17, Steven D'Aprano wrote: On Sat, 23 Jun 2012 19:14:43 +0100, Rotwang wrote: The problem is that if the object was pickled by the module run as a script and then unpickled by the imported module, the unpickler looks in __main__ rather than mymodule for the object's class, and

Re: cPickle - sharing pickled objects between scripts and imports

2012-06-23 Thread Steven D'Aprano
On Sat, 23 Jun 2012 19:14:43 +0100, Rotwang wrote: > The problem is that if the object was > pickled by the module run as a script and then unpickled by the imported > module, the unpickler looks in __main__ rather than mymodule for the > object's class, and doesn't find it. Possibly the solutio

Re: cPickle - sharing pickled objects between scripts and imports

2012-06-23 Thread Rotwang
On 23/06/2012 18:31, Dave Angel wrote: On 06/23/2012 12:13 PM, Peter Otten wrote: Rotwang wrote: Hi all, I have a module that saves and loads data using cPickle, and I've encountered a problem. Sometimes I want to import the module and use it in the interactive Python interpreter, whereas some

Re: cPickle - sharing pickled objects between scripts and imports

2012-06-23 Thread Rotwang
On 23/06/2012 17:13, Peter Otten wrote: Rotwang wrote: Hi all, I have a module that saves and loads data using cPickle, and I've encountered a problem. Sometimes I want to import the module and use it in the interactive Python interpreter, whereas sometimes I want to run it as a script. But obj

Re: cPickle - sharing pickled objects between scripts and imports

2012-06-23 Thread Dave Angel
On 06/23/2012 12:13 PM, Peter Otten wrote: > Rotwang wrote: > >> Hi all, I have a module that saves and loads data using cPickle, and >> I've encountered a problem. Sometimes I want to import the module and >> use it in the interactive Python interpreter, whereas sometimes I want >> to run it as a

Re: cPickle - sharing pickled objects between scripts and imports

2012-06-23 Thread Peter Otten
Rotwang wrote: > Hi all, I have a module that saves and loads data using cPickle, and > I've encountered a problem. Sometimes I want to import the module and > use it in the interactive Python interpreter, whereas sometimes I want > to run it as a script. But objects that have been pickled by runn

Re: cPickle -> invalid signature

2011-05-17 Thread Gabriel Genellina
En Tue, 17 May 2011 15:26:53 -0300, Neal Becker escribió: Gabriel Genellina wrote: En Tue, 17 May 2011 08:41:41 -0300, Neal Becker escribió: What does it mean when cPickle.load says: RuntimeError: invalid signature Is binary format not portable? Are you sure that's the actual error mess

Re: cPickle -> invalid signature

2011-05-17 Thread Gabriel Genellina
En Tue, 17 May 2011 15:26:53 -0300, Neal Becker escribió: Gabriel Genellina wrote: En Tue, 17 May 2011 08:41:41 -0300, Neal Becker escribió: What does it mean when cPickle.load says: RuntimeError: invalid signature Is binary format not portable? Are you sure that's the actual error mess

Re: cPickle -> invalid signature

2011-05-17 Thread Neal Becker
Gabriel Genellina wrote: > En Tue, 17 May 2011 08:41:41 -0300, Neal Becker > escribió: > >> What does it mean when cPickle.load says: >> RuntimeError: invalid signature >> >> Is binary format not portable? > > Are you sure that's the actual error message? > I cannot find such message anywhere i

Re: cPickle -> invalid signature

2011-05-17 Thread Gabriel Genellina
En Tue, 17 May 2011 08:41:41 -0300, Neal Becker escribió: What does it mean when cPickle.load says: RuntimeError: invalid signature Is binary format not portable? Are you sure that's the actual error message? I cannot find such message anywhere in the sources. The pickle format is quite po

Re: cPickle segfault with nested dicts in threaded env

2010-09-09 Thread Kenneth Dombrowski
On Wed, Sep 8, 2010 at 6:55 PM, Paul Rubin wrote: > Carl Banks writes: >> Since Python 2.7 is released, Python 2.5 is no longer accepting bug >> fixes, only security fixes.  So be aware. > > Segfaults should be treated as security holes unless there's convincing > reasons that no exploit is possi

Re: cPickle segfault with nested dicts in threaded env

2010-09-08 Thread Aahz
In article , Thomas Jollans wrote: > >Also, Python 2.5 is frightfully old. [...] "Frightfully"??? I'm sure plenty of people are still using Python 2.3 in production environments (certainly my last job did as of 1.5 years ago, and I would be mildly surprised if they upgraded by now). -- Aahz (

Re: cPickle segfault with nested dicts in threaded env

2010-09-08 Thread Paul Rubin
Carl Banks writes: > Since Python 2.7 is released, Python 2.5 is no longer accepting bug > fixes, only security fixes. So be aware. Segfaults should be treated as security holes unless there's convincing reasons that no exploit is possible. So the bug should be reported against 2.5 as well as l

Re: cPickle segfault with nested dicts in threaded env

2010-09-08 Thread Carl Banks
On Sep 8, 11:58 am, Kenneth Dombrowski wrote: > Hi all, > > Before creating an issue @ bugs.python.org I wanted to run the > following by everyone > > We are having a difficult time with what looks like a cPickle issue > when given a data structure containing >15 nested dictionaries, but > only wh

Re: cPickle segfault with nested dicts in threaded env

2010-09-08 Thread Thomas Jollans
On Wednesday 08 September 2010, it occurred to Kenneth Dombrowski to exclaim: > Environment is FreeBSD 8, Python 2.5.5 Which architecture? Also, Python 2.5 is frightfully old. There's not really any problem with still using it, but nobody's maintaining it upstream, so don't bother reporting a b

Re: cPickle error when caching data

2010-08-03 Thread Benedict Verheyen
On 3/08/2010 17:01, Peter Otten wrote: > You can only pickle instances of classes that are reachable by the import > system as only the qualified name of the class is stored, not the bytecode > to generate it. Move your class out of the function into the global module > scope and you should be

Re: cPickle error when caching data

2010-08-03 Thread Peter Otten
Benedict Verheyen wrote: > i get the following error when trying to set data in the cache of a django > application. The error is however a python error as it involves pickling > and i can reproduce it in a shell. > The error i get is this: > cPickle.PicklingError: Can't pickle : > attribute looku

Re: cPickle and subclassing lists?

2009-04-17 Thread Carl Banks
On Apr 17, 10:13 am, Reckoner wrote: > I have a large class that is a child of list. I need to pickle it, but > it's not working. For example, I have reduced it to the following: > > class Mylist(list): >     def __init__(self,x=[]): >         list.__init__(self,x) > > and I cannot even get this t

Re: cPickle and subclassing lists?

2009-04-17 Thread Reckoner
On Apr 17, 11:16 am, Piet van Oostrum wrote: > > Reckoner (R) wrote: > >R> I have a large class that is a child of list. I need to pickle it, but > >R> it's not working. For example, I have reduced it to the following: > >R> class Mylist(list): > >R> def __init__(self,x=[]): > >R> l

Re: cPickle and subclassing lists?

2009-04-17 Thread Piet van Oostrum
> Reckoner (R) wrote: >R> I have a large class that is a child of list. I need to pickle it, but >R> it's not working. For example, I have reduced it to the following: >R> class Mylist(list): >R> def __init__(self,x=[]): >R> list.__init__(self,x) >R> and I cannot even get this to pi

Re: cPickle and subclassing lists?

2009-04-17 Thread Peter Otten
Reckoner wrote: > On Apr 17, 10:42 am, Peter Otten <__pete...@web.de> wrote: >> Reckoner wrote: >> > I have a large class that is a child of list. I need to pickle it, but >> > it's not working. For example, I have reduced it to the following: >> >> > class Mylist(list): >> > def __init__(self

Re: cPickle and subclassing lists?

2009-04-17 Thread Reckoner
On Apr 17, 10:42 am, Peter Otten <__pete...@web.de> wrote: > Reckoner wrote: > > I have a large class that is a child of list. I need to pickle it, but > > it's not working. For example, I have reduced it to the following: > > > class Mylist(list): > > def __init__(self,x=[]): > > list.__init__

Re: cPickle and subclassing lists?

2009-04-17 Thread Peter Otten
Reckoner wrote: > I have a large class that is a child of list. I need to pickle it, but > it's not working. For example, I have reduced it to the following: > > class Mylist(list): > def __init__(self,x=[]): > list.__init__(self,x) > > and I cannot even get this to pickle right. > >>> w=My

Re: cPickle vs pickle discrepancy

2009-01-08 Thread Zac Burns
Correct, 2.6.1 does not complain. If only I could upgrade! The workaround is obvious and I'll do that. Thanks for your help. -- Zachary Burns (407)590-4814 Aim - Zac256FL Production Engineer (Digital Overlord) Zindagi Games On Thu, Jan 8, 2009 at 11:34 AM, Carl Banks wrote: > On Jan 8, 12:27

Re: cPickle vs pickle discrepancy

2009-01-08 Thread Carl Banks
On Jan 8, 12:27 pm, "Zac Burns" wrote: > Thanks for your patience waiting for me to isolate the problem. > > | Package > --__init__.py ->empty > --Package.py ->empty > --Module.py >      import cPickle >      class C(object): >         pass >      def fail(): >         return cPickle.dumps(C(), -1

Re: cPickle vs pickle discrepancy

2009-01-08 Thread Zac Burns
Thanks for your patience waiting for me to isolate the problem. | Package --__init__.py ->empty --Package.py ->empty --Module.py import cPickle class C(object): pass def fail(): return cPickle.dumps(C(), -1) import Package.Module Package.Module.fail() The failure s

Re: cPickle vs pickle discrepancy

2009-01-06 Thread Gabriel Genellina
En Mon, 05 Jan 2009 23:04:30 -0200, Zac Burns escribió: I have a module that attempts to pickle classes defined in that module. I get an error of the form: PicklingError: Can't pickle : import of module Module.SubModule failed when using cPickle (protocol -1, python version 2.5.1). The module

Re: cPickle

2008-09-04 Thread Maric Michaud
Le Thursday 04 September 2008 13:08:59 Gerhard Häring, vous avez écrit : > gopal mishra wrote: > > I have 3 objects and want to save in one pickle file. > > > > I used cPickle to dump 3 objects in a pkl file > > > > i.e cPickle.dump(object1, fileobject, -1) > > > > cPickle.dump(o

Re: cPickle

2008-09-04 Thread Gerhard Häring
gopal mishra wrote: I have 3 objects and want to save in one pickle file. I used cPickle to dump 3 objects in a pkl file i.e cPickle.dump(object1, fileobject, -1) cPickle.dump(object2, fileobject, -1) cPickle.dump(object3, fileobject, -1) I have chang

Re: cPickle asymptotic performance?

2008-06-12 Thread Hrvoje Niksic
Eric Jonas <[EMAIL PROTECTED]> writes: >> Try gc.disable() before loading the pickle, and gc.enable() after. >> >> > Is cPickle's behavior known to be O(n^2)? >> >> No, but the garbage collector's sometimes is. > > Wow, that totally fixed it -- we went from 1200 seconds to 60 > seconds. 60 seco

Re: cPickle asymptotic performance?

2008-06-12 Thread Eric Jonas
On Thu, 2008-06-12 at 20:57 +0200, Hrvoje Niksic wrote: > Eric Jonas <[EMAIL PROTECTED]> writes: > > > I've done some benchmarking while attempting to serialize my (large) > > graph data structure with cPickle; I'm seeing superlinear performance > > (plotting it seems to suggest n^2 where n is th

Re: cPickle asymptotic performance?

2008-06-12 Thread Calvin Spealman
If you are getting to the point where your data is large enough to really care about the speed of cPickle, then maybe its time you moved past pickles for your storage format? 2.5 includes sqlite, so you could persist them in a nice, indexed table or something. Just a suggestion. On Jun 12

Re: cPickle asymptotic performance?

2008-06-12 Thread Hrvoje Niksic
Eric Jonas <[EMAIL PROTECTED]> writes: > I've done some benchmarking while attempting to serialize my (large) > graph data structure with cPickle; I'm seeing superlinear performance > (plotting it seems to suggest n^2 where n is the number of nodes of my > graph), in the duration of the pickle.dum

Re: cPickle problems

2006-11-18 Thread Paul McGuire
"Jeff Poole" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > So when something tries to retrieve .__getstate__, it returns an empty > string. If I tell it to raise an AttributeException instead, then I > get this message: > > TypeError: a class that defines __slots__ without definin

Re: cPickle problems

2006-11-16 Thread Klaas
Jeff Poole wrote: > Good idea. Well, I did that, and I found out that the object causing > problems is a ParseResults object (a class from PyParsing) and that the > __getstate__ member is in fact an empty string (''). I'm not sure > where this leaves me... The PyParsing code clearly never crea

Re: cPickle problems

2006-11-16 Thread Jeff Poole
Good idea. Well, I did that, and I found out that the object causing problems is a ParseResults object (a class from PyParsing) and that the __getstate__ member is in fact an empty string (''). I'm not sure where this leaves me... The PyParsing code clearly never creates such a member and my cod

Re: cPickle problems

2006-11-16 Thread Gabriel Genellina
At Thursday 16/11/2006 21:48, Jeff Poole wrote: File "/usr/lib/python2.4/pickle.py", line 313, in save rv = reduce(self.proto) File "/usr/lib/python2.4/copy_reg.py", line 83, in _reduce_ex dict = getstate() TypeError: 'str' object is not callable Someway, self.__getstate__ is a st

Re: cPickle problems

2006-11-16 Thread Jeff Poole
To clutter this up with yet another message, this is what happens if I use pickle instead of cPickle. Same error, but it has more of a stack trace so someone more advanced than myself might be able to pick out what is going amiss. Traceback (most recent call last): File "./generateTools.py", li

Re: cPickle problems

2006-11-16 Thread Jeff Poole
Oh, and I suppose I should provide some version information: $ python Python 2.4.3 (#1, May 18 2006, 07:40:45) [GCC 3.3.3 (cygwin special)] on cygwin Jeff Poole wrote: > This is going to be a pretty vague message because it involves a large > block of code I'd rather avoid posting. Basically,

Re: Cpickle module... not in Lib installs

2005-03-25 Thread Marcus Lowland
Arrghh, I forgot about case sensitivities sorry about that guys, I'm so used to not having to think about that. Thanks for having patience with me and thanks for the explanations. import cPickle hehe... damn... tricky getting used to that :-) Thanks again, Marcus -- http://mail.python.

Re: Cpickle module... not in Lib installs

2005-03-25 Thread John Machin
Marcus Lowland wrote: > Hello, I'm fairly new to python and have read about and wanted to begin > experimenting with cpickle. As I understand, this should be a native > module in the python library. I have python 2.3 and now just installed > 2.4, but am not able to import or find cpickle.py in any

Re: Cpickle module... not in Lib installs

2005-03-25 Thread Tim Jarman
Marcus Lowland wrote: > Thanks Marc, but... I've searched the file directories for cpickle (not > specifying file type) and only came up with test_cpickle.py. Also, if > cPickle.so were the correct file and existed in my lib then the > following would not happen. > import cpickle > > Traceb

Re: Cpickle module... not in Lib installs

2005-03-25 Thread Marcus Lowland
Thanks Marc, but... I've searched the file directories for cpickle (not specifying file type) and only came up with test_cpickle.py. Also, if cPickle.so were the correct file and existed in my lib then the following would not happen. >>> import cpickle Traceback (most recent call last): File ""

Re: Cpickle module... not in Lib installs

2005-03-25 Thread Marc 'BlackJack' Rintsch
In <[EMAIL PROTECTED]>, Marcus Lowland wrote: > Hello, I'm fairly new to python and have read about and wanted to begin > experimenting with cpickle. As I understand, this should be a native > module in the python library. I have python 2.3 and now just installed > 2.4, but am not able to import o