>
> Anywaymarshalshould not be used by user code to serialize objects.
> It's only meant for Python byte code. Please use the pickle/cPickle
> module instead.
>
> Christian
Just for yucks let me point out that marshal has
no real security concerns of interest to the non-paranoid,
whereas pickle i
On Jun 15, 8:08 am, [EMAIL PROTECTED] wrote:
> Indeed. I (the OP) am using a production release which has the 1k
> linear growth.
> I am seeing the problems with ~5MB and ~10MB sizes.
> Apparently this will be improved greatly in Python 2.6, at least up to
> the 32MB limit.
I've just fixed this fo
On Jun 16, 1:08 am, [EMAIL PROTECTED] wrote:
> On Jun 15, 3:16 am, John Machin <[EMAIL PROTECTED]> wrote:
>
> > But that change went into the svn trunk on 11-May-2008; perhaps the OP
> > is using a production release which would have the previous version,
> > which is merely "newsize = size + 1024;
On Jun 15, 3:16 am, John Machin <[EMAIL PROTECTED]> wrote:
> But that change went into the svn trunk on 11-May-2008; perhaps the OP
> is using a production release which would have the previous version,
> which is merely "newsize = size + 1024;".
>
> Do people really generate 32MB pyc files, or is
Raymond Hettinger wrote:
> When more space is needed, the resize operation over-allocates by
> double the previous need plus 1K. This should give amortized O(1)
> performance just like list.append().
>
> However, when that strategy requests more than 32Mb, the resizing
> becomes less aggressive a
John Machin wrote:
>> Here's how marshal resizes the string:
>>
>> newsize = size + size + 1024;
>> if (newsize > 32*1024*1024) {
>> newsize = size + 1024*1024;
>> }
>>
>> Maybe you can split your large objects and marshal multiple objects to
>> keep the siz
On Jun 15, 7:47 pm, Peter Otten <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote:
> > I'm stuck on a problem where I want to use marshal for serialization
> > (yes, yes, I know (c)Pickle is normally recommended here). I favor
> > marshal for speed for the types of data I use.
>
> > However it s
On Jun 15, 1:04 am, [EMAIL PROTECTED] wrote:
> However it seems that marshal.dumps() for large objects has a
> quadratic performance issue which I'm assuming is that it grows its
> memory buffer in constant increments.
Looking at the source in
http://svn.python.org/projects/python/trunk/Python/ma
[EMAIL PROTECTED] wrote:
> I'm stuck on a problem where I want to use marshal for serialization
> (yes, yes, I know (c)Pickle is normally recommended here). I favor
> marshal for speed for the types of data I use.
>
> However it seems that marshal.dumps() for large objects has a
> quadratic perfo
On 16:04, domenica 15 giugno 2008 [EMAIL PROTECTED] wrote:
> cStringIO.StringIO object to marshal.dump() instead but I quickly
> learned this is not supported (only true file objects are supported).
>
> Any ideas about how to get around the marshal quadratic issue? Any
> hope for a fix for that
10 matches
Mail list logo
