Re: Strange occasional marshal error

2011-03-03 Thread Graham Stratton
On Mar 3, 6:04 pm, Guido van Rossum wrote: > This bug report doesn't mention the Python version nor the platform -- > it could in theory be a bug in the platform compiler or memory > allocator. I've seen the problem with 2.6 and 2.7, on RHEL 4 (possibly with a custom kernel, I can't check at the

Re: [Python-Dev] Strange occasional marshal error

2011-03-03 Thread Guido van Rossum
On Thu, Mar 3, 2011 at 9:40 AM, MRAB wrote: > On 03/03/2011 15:09, Graham Stratton wrote: >> >> On Mar 2, 3:01 pm, Graham Stratton  wrote: >>> >>> We are using marshal for serialising objects before distributing them >>> around the cluster, and extremely occasionally a corrupted marshal is >>> pro

Re: Strange occasional marshal error

2011-03-03 Thread MRAB
On 03/03/2011 15:09, Graham Stratton wrote: On Mar 2, 3:01 pm, Graham Stratton wrote: We are using marshal for serialising objects before distributing them around the cluster, and extremely occasionally a corrupted marshal is produced. The current workaround is to serialise everything twice and

Re: Strange occasional marshal error

2011-03-03 Thread Tom Zych
On Thu, 03 Mar 2011 07:09 -0800, "Graham Stratton" wrote: > Every corruption point occurs exactly three bytes above an extension > point (rounded to the nearest word for the last two). This clearly > isn't a coincidence, but I can't see where there could be a problem. > I'd be grateful for any poi

Re: Strange occasional marshal error

2011-03-03 Thread Graham Stratton
On Mar 2, 3:01 pm, Graham Stratton wrote: > We are using marshal for serialising objects before distributing them > around the cluster, and extremely occasionally a corrupted marshal is > produced. The current workaround is to serialise everything twice and > check that the serialisations are the

Strange occasional marshal error

2011-03-02 Thread Graham Stratton
Hi, I'm using Python with ZeroMQ to distribute data around an HPC cluster. The results have been good apart from one issue which I am completely stuck with: We are using marshal for serialising objects before distributing them around the cluster, and extremely occasionally a corrupted marshal is