RE: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7

2024-09-03 Thread AVI GROSS via Python-list
tware or interest. -Original Message- From: Tutor On Behalf Of Alan Gauld via Tutor Sent: Tuesday, September 3, 2024 4:41 AM To: tu...@python.org Cc: python-list@python.org Subject: Re: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found

Re: Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7

2024-09-03 Thread geodandw via Python-list
On 9/2/24 11:36, Barry Scott wrote: On 2 Sep 2024, at 15:00, marc nicole via Python-list wrote: I am using Python 2.7 on Windows 10 Why? Install Python 3.12 and it will be easier to get help and support. If you have legacy that still needs porting then you can install 3.12 along side the

Re: [Tutor] Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7

2024-09-03 Thread marc nicole via Python-list
ore or after the error message? It might make things easier to > debug(clearer error traceback) if you put the code to create the thread > into a separate function? > > def do_Something(text)... > > def start(fn): >q = Process >q.start() > > if __name_ >

Re: Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7

2024-09-02 Thread Barry Scott via Python-list
> On 2 Sep 2024, at 15:00, marc nicole via Python-list > wrote: > > I am using Python 2.7 on Windows 10 Why? Install Python 3.12 and it will be easier to get help and support. If you have legacy that still needs porting then you can install 3.12 along side the unsupported 3.12. Barry --

Getting a Process.start() error pickle.PicklingError: Can't pickle : it's not found as __builtin__.module with Python 2.7

2024-09-02 Thread marc nicole via Python-list
PicklingError: Can't pickle : it's not found as __builtin__.module anybody could provide an alternative to call the function do_something() in a separate thread ? -- https://mail.python.org/mailman/listinfo/python-list

Re: Python-pickle error

2023-05-09 Thread Tony Flury via Python-list
Charles, by your own admission, you deleted your pkl file, And your code doesn't write that pkl file (pickle.dumps(...) doesn't write a file it creates a new string and at no point will it write to the file : What you need is this : import pickle

Re: Python-pickle error

2023-04-19 Thread Thomas Passin
On 4/19/2023 12:14 PM, charles wiewiora wrote: Hello, I am experincing problems with the pickle moducle the folowing code was working before, import pickle number=2 my_pickeld_object=pickle.dumps(number) print("this is my pickled object",{my_pickeld_object},) with open('file.pkl&

Python-pickle error

2023-04-19 Thread charles wiewiora
Hello, I am experincing problems with the pickle moducle the folowing code was working before, import pickle number=2 my_pickeld_object=pickle.dumps(number) print("this is my pickled object",{my_pickeld_object},) with open('file.pkl', 'rb') as fil

Re: Pickle file and send via socket

2022-02-18 Thread Chris Angelico
On Sat, 19 Feb 2022 at 05:47, UTKARSH PANDEY wrote: > > On Wednesday, August 8, 2012 at 8:37:33 PM UTC+5:30, lipska the kat wrote: > > ... > Directly read bytes from file and send it over the socket object from client > side in while loop until all content from file is read. > Almost ten years.

Re: Pickle file and send via socket

2022-02-18 Thread UTKARSH PANDEY
.org/py3k/tutorial/inputoutput.html#reading-and-writing-files > > > it should be straightforward to read and write pickled files > Not sure why you want to pickle a text file over the network when you > could just stream it between ports ! > > however ... > > I'm

Re: Pickle segfaults with custom type

2022-01-15 Thread Marco Sulla
the pickling of views and iter types. Python segfaults if I try to pickle > them. > > For example, I have: > > > static PyTypeObject PyFrozenDictIterKey_Type = { > PyVarObject_HEAD_INIT(NULL, 0) > "frozendict.keyiterator",

Pickle segfaults with custom type

2022-01-07 Thread Marco Sulla
I have a custom implementation of dict using a C extension. All works but the pickling of views and iter types. Python segfaults if I try to pickle them. For example, I have: static PyTypeObject PyFrozenDictIterKey_Type = { PyVarObject_HEAD_INIT(NULL, 0) "frozendict.keyite

Re: recover pickled data: pickle data was truncated

2022-01-05 Thread iMath
Thanks for all your kind help, wish you a promising year! -- https://mail.python.org/mailman/listinfo/python-list

Re: recover pickled data: pickle data was truncated

2022-01-02 Thread Barry Scott
econd. Oh and yes use JSON, it is far better as a way of exchanging data than pickle. Easy to read and check, can be processes in many languages. Barry > > On Sat, 1 Jan 2022 at 14:11, Barry wrote: >> >> >> >>> On 31 Dec 2021, at 17:53, iMath wrote: >>

Re: recover pickled data: pickle data was truncated

2022-01-01 Thread Marco Sulla
ry, Chris and Avi. IMHO your data is lost. Unpickling > >> it by hand is a harsh work and maybe unreliable. > >> > >> Is there any reason you can't simply add a semaphore to avoid writing > >> at the same time and re-run the code and regenerate the data? > >

Re: recover pickled data: pickle data was truncated

2022-01-01 Thread Barry
and re-run the code and regenerate the data? > > Thanks for your replies! I didn't have a sense of adding a semaphore on > writing to pickle data before, so corrupted the data. > Since my data was colleted in the daily usage, so cannot re-run the code and > regenerate the dat

Re: recover pickled data: pickle data was truncated

2021-12-31 Thread iMath
a sense of adding a semaphore on writing to pickle data before, so corrupted the data. Since my data was colleted in the daily usage, so cannot re-run the code and regenerate the data. In order to avoid corrupting my data again and the complicity of using a semaphore, now I am using json text

Re: recover pickled data: pickle data was truncated

2021-12-29 Thread Marco Sulla
On Wed, 29 Dec 2021 at 18:33, iMath wrote: > But I found the size of the file of the shelve data didn't change much, so I > guess the data are still in it , I just wonder any way to recover my data. I agree with Barry, Chris and Avi. IMHO your data is lost. Unpickling it by hand is a harsh work

RE: recover pickled data: pickle data was truncated

2021-12-29 Thread Avi Gross via Python-list
checking it. But corruption can happen for many reasons including at the level of the disk it is written to. -Original Message- From: Python-list On Behalf Of iMath Sent: Wednesday, December 29, 2021 10:51 AM To: python-list@python.org Subject: Re: recover pickled data: pickle data was

Re: recover pickled data: pickle data was truncated

2021-12-29 Thread Chris Angelico
ersions got interleaved, in which case I strongly advise you NOT to try unpickling it. If you really feel like delving into it, try manually decoding the pickle stream, but be very careful. ChrisA -- https://mail.python.org/mailman/listinfo/python-list

Re: recover pickled data: pickle data was truncated

2021-12-29 Thread iMath
> You have lost the data in that case. But I found the size of the file of the shelve data didn't change much, so I guess the data are still in it , I just wonder any way to recover my data. -- https://mail.python.org/mailman/listinfo/python-list

Re: recover pickled data: pickle data was truncated

2021-12-26 Thread Barry Scott
write by two >> processes, thus corrupted it. Is there any way to recover all data in it ? >> Currently I just get "pickle data was truncated" exception after reading a >> portion of the data? You have lost the data in that case. You will need to do what Marco sugge

Re: recover pickled data: pickle data was truncated

2021-12-26 Thread Marco Sulla
l data in it ? > Currently I just get "pickle data was truncated" exception after reading a > portion of the data? > > Data and code here > :https://drive.google.com/file/d/137nJFc1TvOge88EjzhnFX9bXg6vd0RYQ/view?usp=sharing > -- > https://mail.python.org/mailman/listinfo/pyth

recover pickled data: pickle data was truncated

2021-12-25 Thread iMath
Normally, the shelve data should be read and write by only one process at a time, but unfortunately it was simultaneously read and write by two processes, thus corrupted it. Is there any way to recover all data in it ? Currently I just get "pickle data was truncated" exception after

Re: Pickle in C does not work

2020-10-20 Thread Serhiy Storchaka
20.10.20 17:12, Marco Sulla пише: > On Tue, 20 Oct 2020 at 16:07, Serhiy Storchaka wrote: >> You can use PyDict_New() + PyDict_Merge() to create a dict from your >> mapping. > Well, yes, I know. I just wrote it for simplicity now. Do you think this is > the problem? I think that it can be more pe

Re: Pickle in C does not work

2020-10-20 Thread Marco Sulla
On Tue, 20 Oct 2020 at 16:12, Marco Sulla wrote: > tp_name of your class should include the module name. >> > > It will be done :-) > It was the name... Thank you a lot! -- https://mail.python.org/mailman/listinfo/python-list

Re: Pickle in C does not work

2020-10-20 Thread Marco Sulla
On Tue, 20 Oct 2020 at 16:07, Serhiy Storchaka wrote: > You can use PyDict_New() + PyDict_Merge() to create a dict from your > mapping. > Well, yes, I know. I just wrote it for simplicity now. Do you think this is the problem? I forgot to say that copy and deepcopy works. For what I know, they u

Re: Pickle in C does not work

2020-10-20 Thread Serhiy Storchaka
20.10.20 01:28, Marco Sulla пише: > PyObject *d = PyObject_Call((PyObject *)&PyDict_Type, args, NULL); You can use PyDict_New() + PyDict_Merge() to create a dict from your mapping. > but I get: > _pickle.PicklingError: Can't pickle : attribute lookup > frozendict on bui

Pickle in C does not work

2020-10-19 Thread Marco Sulla
*)&PyDict_Type, args, NULL); Py_INCREF(mp); return Py_BuildValue("O(N)", Py_TYPE(mp), d); } but I get: _pickle.PicklingError: Can't pickle : attribute lookup frozendict on builtins failed The identical (?) Python code works: def __reduce__(self, *args, **kwargs): ret

Re: Pickle caching objects?

2019-12-16 Thread Peter J. Holzer
On 2019-12-16 08:56:26 +0100, dieter wrote: > Note also that Python memeory management is quite elaborate: > not every memory block is immediately obtained from and released > to the operating system: Python has its own memory management > data structures (to fill the gap between the fine grained >

Re: Pickle caching objects?

2019-12-16 Thread Inada Naoki
on the IRC channel but didn't manage to get a > response, though some people replied with suggestions that expanded this > question a bit. > > I have a program that has to read some pickle files, perform some > operations on them, and then return. The pickle objects I am reading all

Re: Pickle caching objects?

2019-12-15 Thread dieter
José María Mateos writes: > I just asked this question on the IRC channel but didn't manage to get > a response, though some people replied with suggestions that expanded > this question a bit. > > I have a program that has to read some pickle files, perform some > opera

Re: Pickle caching objects?

2019-12-01 Thread José María Mateos
On Sun, Dec 01, 2019 at 12:26:15PM +1100, Chris Angelico wrote: I can't answer your question authoritatively, but I can suggest a place to look. Python's memory allocator doesn't always return memory to the system when the objects are freed up, for various reasons including the way that memory pa

Re: Pickle caching objects?

2019-11-30 Thread Richard Damon
On 11/30/19 5:05 PM, José María Mateos wrote: > Hi, > > I just asked this question on the IRC channel but didn't manage to get > a response, though some people replied with suggestions that expanded > this question a bit. > > I have a program that has to read some

Re: Pickle caching objects?

2019-11-30 Thread Chris Angelico
uld expect it to. But > the loading time is constant, so I think I can disregard any pickle > caching mechanisms. > > So I guess now my question is: can anyone give me any pointers as to why > is this happening? Any help is appreciated. > I can't answer your question authoritat

Pickle caching objects?

2019-11-30 Thread José María Mateos
Hi, I just asked this question on the IRC channel but didn't manage to get a response, though some people replied with suggestions that expanded this question a bit. I have a program that has to read some pickle files, perform some operations on them, and then return. The pickle obje

Re: Pickle failed __getstate__ on my customized class inherited dict

2019-11-20 Thread Peter Otten
lampahome wrote: > I make a class Wrapper inherited from dict and met problem when I want to > pickle it. > > Is there anyway to make __getstate__ of Wrapper to output a normal > dict?(Output a dict will help pickleing easily) > > > === code === > import pickle >

Re: Pickle failed __getstate__ on my customized class inherited dict

2019-11-19 Thread dieter
lampahome writes: > I make a class Wrapper inherited from dict and met problem when I want to > pickle it. > > Is there anyway to make __getstate__ of Wrapper to output a normal > dict?(Output a dict will help pickleing easily) > > > === code === > import pickle >

Pickle failed __getstate__ on my customized class inherited dict

2019-11-19 Thread lampahome
I make a class Wrapper inherited from dict and met problem when I want to pickle it. Is there anyway to make __getstate__ of Wrapper to output a normal dict?(Output a dict will help pickleing easily) === code === import pickle class Wrapper(dict): def __getattr__(self, attr): return

Re: in a pickle

2019-03-06 Thread duncan smith
attempt to unpickle.) >> >> A relatively minimal example is pasted below. >> >> >> --> import pickle >> --> class test(dict): >> ...    def __init__(self, keys, shape=None): >> ...    self.shape = shape >> ...    for

Re: in a pickle

2019-03-06 Thread duncan smith
y with the >>> pickling, but I get the error when I attempt to unpickle.) >>> >>> A relatively minimal example is pasted below. >>> >>> >>>>>> import pickle >>>>>> class test(dict): >>> def __init__(self,

Re: in a pickle

2019-03-06 Thread Ethan Furman
; import pickle --> class test(dict): ...def __init__(self, keys, shape=None): ...self.shape = shape ...for key in keys: ...self[key] = None ...def __setitem__(self, key, val): ...print (self.shape) ...dict.__setitem__(self, key, val) ... -

Re: in a pickle

2019-03-06 Thread Peter Otten
o unpickle.) >> >> A relatively minimal example is pasted below. >> >> >>>>> import pickle >>>>> class test(dict): >> def __init__(self, keys, shape=None): >> self.shape = shape >> for key in keys: >> sel

Re: in a pickle

2019-03-06 Thread duncan smith
nimal example is pasted below. > > >>>> import pickle >>>> class test(dict): > def __init__(self, keys, shape=None): > self.shape = shape > for key in keys: > self[key] = None > >

RE: in a pickle

2019-03-06 Thread David Raymond
tored yet while it's trying to set self[1], which is why trying to print self.shape is failing. Here's a modified version with more prints to show some of that: import pickle class test(dict): def __init__(self, keys, shape = None): print("__init__ is running

Re: in a pickle

2019-03-06 Thread duncan smith
[snip] Sorry, this is Python 3.6 on Linux. Duncan -- https://mail.python.org/mailman/listinfo/python-list

in a pickle

2019-03-06 Thread duncan smith
Hello, I've been trying to figure out why one of my classes can be pickled but not unpickled. (I realise the problem is probably with the pickling, but I get the error when I attempt to unpickle.) A relatively minimal example is pasted below. >>> import pickle >&

Re: pickle and module versioning

2018-12-18 Thread dieter
Neal Becker writes: > ... > Has anyone else run into this issue People who use the "ZODB" (= "Zope Object DataBase") are familiar with it. > and have any ideas? There are things, you must not do. In the "ZODB" case, for example, you must not change whether the class inherits from "persistent.Pe

Re: pickle and module versioning

2018-12-17 Thread Peter Otten
Neal Becker wrote: > I find pickle really handy for saving results from my (simulation) > experiments. But recently I realized there is an issue. Reading the > saved results requires loading the pickle, which in turn will load any > referenced > modules. Problem is, what if t

RE: pickle and module versioning

2018-12-17 Thread Calvin Spealman
This can definitely be a problem, but there are tried and true strategies to deal with it and the best course of action depends a lot on what works for you. Pickle deserialization is just an API instantiating your class instances. That means all the same rules and approaches to API changes

pickle and module versioning

2018-12-17 Thread Neal Becker
I find pickle really handy for saving results from my (simulation) experiments. But recently I realized there is an issue. Reading the saved results requires loading the pickle, which in turn will load any referenced modules. Problem is, what if the modules have changed? For example, I just

Re: [ZODB] Problem with pickle in zodb

2018-06-04 Thread Etienne Robillard
ile "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 91, in pickle_decode return unpickler.load() # msgid, flags, name, args ValueError: unsupported pickle protocol: 3 07:32:52 ZEO.asyncio.client Registration or cache validation failed, Closed Traceback (most recent ca

Problem with pickle in zodb

2018-06-03 Thread Etienne Robillard
self.decode(data)   File "/home/erob/work/ZEO-5.1.0/src/ZEO/asyncio/marshal.py", line 91, in pickle_decode     return unpickler.load() # msgid, flags, name, args ValueError: unsupported pickle protocol: 3 07:32:52 ZEO.asyncio.client   Registration or cache validation failed, Closed Tr

Re: Regarding the error: TypeError: can�t pickle _thread.lock objects

2018-01-04 Thread dieter
e CMU Sphinx website) > > Following the above procedure, I encountered the following error: TypeError: > can.t pickle _thread.lock objects-please find the attached screenshot for > your reference. This looks like a programming error in "g2p-seq2sec": contact its authors or try

Re: Regarding the error: TypeError: can’t pickle _thread.lock objects

2017-12-21 Thread Terry Reedy
procedure, I encountered the following error: TypeError: can’t pickle _thread.lock objects-please find the attached screenshot for your reference. This is a text-only mailing list. One should copy and paste an entire traceback. That said, if Tensorflow or g2p are trying to pickle something

Regarding the error: TypeError: can’t pickle _thread.lock objects

2017-12-21 Thread Winston Manuel Vijay
: TypeError: can’t pickle _thread.lock objects-please find the attached screenshot for your reference. Thanks, A.Winston Manuel Vijay This e-mail and any files transmitted with it are for the sole use of the intended recipient(s) and may contain confidential and

Why is it faster to first pickle data before sending a dict than sending a dict directly in Python 3?

2017-06-12 Thread Martin Bammer
Following test program: import time import marshal try: import cPickle as pickle str_ = unicode except: import pickle str_ = str def TestPrc(rd): d = rd.recv() print("OK") def TestPrc2(rd): d = pickle.loads(rd.recv()) print("OK") if _

Re: dictionary of pickle error when get it in multiprocessing and has name error

2017-02-27 Thread Peter Otten
Ho Yeung Lee wrote: > before cloususerlogin > Unexpected error: > after map pool > > > ... > passwordlist = pickle.load( open( str(currentworkingdirectory) + > "\\decryptedsecret.p", "rb" ) ) According to https://docs.python.org/dev/library/multiprocessing.html#programming-guidelines you cann

dictionary of pickle error when get it in multiprocessing and has name error

2017-02-26 Thread Ho Yeung Lee
before cloususerlogin Unexpected error: after map pool ... passwordlist = pickle.load( open( str(currentworkingdirectory) + "\\decryptedsecret.p", "rb" ) ) def processInput(host): try: decrypt_file(sys.argv[1], str(currentworkingdirectory) + "\\encryptedsecret.p", str(currentworki

Re: MemoryError and Pickle

2016-11-21 Thread Steve D'Aprano
On Tue, 22 Nov 2016 10:27 am, Fillmore wrote: > > Hi there, Python newbie here. > > I am working with large files. For this reason I figured that I would > capture the large input into a list and serialize it with pickle for > later (faster) usage. > Everything has wor

Re: MemoryError and Pickle

2016-11-21 Thread Steve D'Aprano
On Tue, 22 Nov 2016 11:40 am, Peter Otten wrote: > Fillmore wrote: > >> Hi there, Python newbie here. >> >> I am working with large files. For this reason I figured that I would >> capture the large input into a list and serialize it with pickle for >>

Re: MemoryError and Pickle

2016-11-21 Thread Chris Kaynor
eObject.write("wa: %s\n" % parts[1]) > fileObject.write("ua: %s\n" % parts[2]) > fileObject.close() > > But this doesn't use pickle format, so your reader program would have to > be modified to read this format. And you'll run into the same problem if > the

Re: MemoryError and Pickle

2016-11-21 Thread Peter Otten
Fillmore wrote: > Hi there, Python newbie here. > > I am working with large files. For this reason I figured that I would > capture the large input into a list and serialize it with pickle for > later (faster) usage. But is it really faster? If the pickle is, let's say, t

Re: MemoryError and Pickle

2016-11-21 Thread John Gordon
ename, "w") for line in sys.stdin: parts = line.strip().split("\t") fileObject.write("ta: %s\n" % parts[0]) fileObject.write("wa: %s\n" % parts[1]) fileObject.write("ua: %s\n" % parts[2]) fileObject.close() But this d

MemoryError and Pickle

2016-11-21 Thread Fillmore
Hi there, Python newbie here. I am working with large files. For this reason I figured that I would capture the large input into a list and serialize it with pickle for later (faster) usage. Everything has worked beautifully until today when the large data (1GB) file caused a MemoryError

Re: problem using pickle

2016-10-27 Thread Veek M
Ben Finney wrote: > "Veek. M" writes: > >> class Foo(object): >> pass >> >> object is a keyword and you're using it as an identifier > > Python does not have ‘object’ as a keyword. ‘and’ is a keyword. > > Here's the difference:: > > >>> object > > >>> object = "Lorem ipsum" >

Re: problem using pickle

2016-10-27 Thread Veek M
Rustom Mody wrote: > On Saturday, July 2, 2016 at 9:17:01 AM UTC+5:30, Veek. M wrote: >> object is a keyword and you're using it as an identifier > > keyword and builtin are different > In this case though the advice remains the same > In general maybe not... > Just sayin' np - feel free to corre

Can't pickle : attribute lookup __builtin__.gen erator failed

2016-10-13 Thread meInvent bbird
Traceback (most recent call last): File "C:\Python27\lib\multiprocessing\queues.py", line 262, in _feed send(obj) PicklingError: Can't pickle : attribute lookup __builtin__.gen erator failed #python pusher.py tcp://*:8080 import sys import time import zmq from multiprocessi

Re: Using copyreg to pickle unpicklable oblects

2016-09-19 Thread Peter Otten
to convert it to a picklable >> > format, I can run: >> > >> > sound_object.get_raw() >> > >> > and to turn that back into an object, I can run: >> > >> > sound_object = pygame.mixer.Sound(raw_data) >> > >> > Is it possible to us

Re: Using copyreg to pickle unpicklable oblects

2016-09-19 Thread kerbingamer376
> > sound_object.get_raw() > > > > and to turn that back into an object, I can run: > > > > sound_object = pygame.mixer.Sound(raw_data) > > > > Is it possible to use copyreg or something similar so it's done > > automatically when I

Re: Using copyreg to pickle unpicklable oblects

2016-09-19 Thread Peter Otten
pygame.mixer.Sound(raw_data) > > Is it possible to use copyreg or something similar so it's done > automatically when I pickle pygame.mixer.Sound() objects? Have a look at the example from the documentation: >>> import copyreg, copy, pickle >>> class C(obj

Using copyreg to pickle unpicklable oblects

2016-09-19 Thread kerbingamer376
cally when I pickle pygame.mixer.Sound() objects? -- https://mail.python.org/mailman/listinfo/python-list

Re: problem using pickle

2016-07-01 Thread Ben Finney
"Veek. M" writes: > class Foo(object): > pass > > object is a keyword and you're using it as an identifier Python does not have ‘object’ as a keyword. ‘and’ is a keyword. Here's the difference:: >>> object >>> object = "Lorem ipsum" >>> object 'Lorem ipsum' >>> and

Re: problem using pickle

2016-07-01 Thread Rustom Mody
On Saturday, July 2, 2016 at 9:17:01 AM UTC+5:30, Veek. M wrote: > object is a keyword and you're using it as an identifier keyword and builtin are different In this case though the advice remains the same In general maybe not... Just sayin' -- https://mail.python.org/mailman/listinfo/python-list

Re: problem using pickle

2016-07-01 Thread Veek. M
Nicky Mac wrote: > Dear Python team, > I have studied the excellent documentation, and attempted to make use > of pickle thus: > > filename = 'my_saved_adventure' > import pickle > class object: > def __init__(self,i,.t) : > self.id

Re: Cassandra multiprocessing can't pickle _thread.lock objects

2016-06-21 Thread dieter
Daiyue Weng writes: > ... > I tried to use Cassandra and multiprocessing to insert rows (dummy data) > concurrently based on the examples in > ... > self.pool = Pool(processes=process_count, initializer=self._setup, > initargs=(session,)) > > I am wondering how to resolve

RE: Cassandra multiprocessing can't pickle _thread.lock objects

2016-06-21 Thread Joaquin Alzola
>I tried to use Cassandra and multiprocessing to insert rows (dummy data) >concurrently based on the examples in Same situation here. I was going to try this solution but on the reading side. Was even thinking of putting Spark-Cassandra just to avoid Python to do multiprocessing. This email is c

Cassandra multiprocessing can't pickle _thread.lock objects

2016-06-21 Thread Daiyue Weng
, processes) params = list() for row in range(10): key = 'test' + str(row) params.append([key, 'test', 'test', 'test', 'test']) start = time.time() rows = qm.get_results(params) delta = time.time() - start

Re: A pickle problem!

2016-04-24 Thread Fabien
On 04/21/2016 11:43 PM, Paulo da Silva wrote: class C(pd.DataFrame): Note also that subclassing pandas is not always encouraged: http://pandas.pydata.org/pandas-docs/stable/internals.html#subclassing-pandas-data-structures Cheers, Fabien -- https://mail.python.org/mailman/listinfo/python-lis

Re: A pickle problem!

2016-04-22 Thread Paulo da Silva
Às 21:33 de 22-04-2016, Ian Kelly escreveu: > On Fri, Apr 22, 2016 at 2:21 PM, Paulo da Silva > wrote: ... > > If they start with two underscores then you could use the name > mangling to find them. If the class name is MyClass then look for any > keys in the instance dict that start with '_MyCla

Re: A pickle problem!

2016-04-22 Thread Ian Kelly
ary because the DataFrame class is already >> customizing its pickle behavior without taking into account the >> possibility of added attributes by subclasses. I think that your >> solution of wrapping the state of the superclass looks fine. >> > Thank you. > > For an

Re: A pickle problem!

2016-04-22 Thread Paulo da Silva
Às 17:27 de 22-04-2016, Ian Kelly escreveu: > On Thu, Apr 21, 2016 at 7:52 PM, Paulo da Silva > wrote: >> Às 22:43 de 21-04-2016, Paulo da Silva escreveu: ... > > Probably this is necessary because the DataFrame class is already > customizing its pickle behavior without ta

Re: A pickle problem!

2016-04-22 Thread Ian Kelly
On Thu, Apr 21, 2016 at 7:52 PM, Paulo da Silva wrote: > Às 22:43 de 21-04-2016, Paulo da Silva escreveu: >> Hi. >> >> Why in this code fragment self.__name is not kept between pickle >> dumps/loads? How to fix it? >> >> Thanks. >> >> import

Re: A pickle problem!

2016-04-21 Thread Paulo da Silva
Às 22:43 de 21-04-2016, Paulo da Silva escreveu: > Hi. > > Why in this code fragment self.__name is not kept between pickle > dumps/loads? How to fix it? > > Thanks. > > import pickle > import pandas as pd > import numpy as np > > class C(pd.DataFrame): &

A pickle problem!

2016-04-21 Thread Paulo da Silva
Hi. Why in this code fragment self.__name is not kept between pickle dumps/loads? How to fix it? Thanks. import pickle import pandas as pd import numpy as np class C(pd.DataFrame): def __init__(self,name,*a,**b): super(C,self).__init__(*a,**b) self

Re: problem using pickle

2016-03-13 Thread Terry Reedy
On 3/12/2016 10:45 AM, Nicky Mac wrote: class object: def __init__(self,i, This is a syntax error, so this is not the code that ran without error. As Peter Otten said, please write a minimal complete runnable example that shows the error and then copy and paste it as run. -- Terry J

Re: problem using pickle

2016-03-13 Thread Peter Otten
Nicky Mac wrote: > Dear Python team, > I have studied the excellent documentation, That is always a laudable endeavour ;) > and attempted to make use of > pickle thus: > > filename = 'my_saved_adventure' > import pickle > class object: 'object' al

problem using pickle

2016-03-13 Thread Nicky Mac
Dear Python team, I have studied the excellent documentation, and attempted to make use of pickle thus: filename = 'my_saved_adventure' import pickle class object: def __init__(self,i,.t) : self.id = i . class world: def __i

Re: Pickle __getstate__ __setstate__ and restoring n/w - beazley pg 172

2016-03-10 Thread dieter
"Veek. M" writes: > ... > what i wanted to know was, x = Client('192.168.0.1') will create an > object 'x' with the IP inside it. When I do: > pickle.dump(x) > pickle doesn't know where in the object the IP is, so he'll call > __getst

Re: Pickle __getstate__ __setstate__ and restoring n/w - beazley pg 172

2016-03-09 Thread Veek. M
Ian Kelly wrote: > On Wed, Mar 9, 2016 at 2:14 AM, Veek. M wrote: >> what i wanted to know was, x = Client('192.168.0.1') will create an >> object 'x' with the IP inside it. When I do: >> pickle.dump(x) >> pickle doesn't know where in the

Re: Pickle __getstate__ __setstate__ and restoring n/w - beazley pg 172

2016-03-09 Thread Ian Kelly
On Wed, Mar 9, 2016 at 2:14 AM, Veek. M wrote: > what i wanted to know was, x = Client('192.168.0.1') will create an > object 'x' with the IP inside it. When I do: > pickle.dump(x) > pickle doesn't know where in the object the IP is, so he'll call > _

Re: Pickle __getstate__ __setstate__ and restoring n/w - beazley pg 172

2016-03-09 Thread Veek. M
; >> def __getstate__(self): >> return self.server_addr >> >> def __setstate__(self,value): >> self.server_addr = value >> self.sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM) >> self.sock.connect(self.server_addr) >> ---

Re: Pickle __getstate__ __setstate__ and restoring n/w - beazley pg 172

2016-03-09 Thread dieter
ver_addr > > def __setstate__(self,value): > self.server_addr = value > self.sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM) > self.sock.connect(self.server_addr) > - > > We'd use it like so: > x = Client(192.168.0.1) > import p

Pickle __getstate__ __setstate__ and restoring n/w - beazley pg 172

2016-03-07 Thread Veek. M
self.sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM) self.sock.connect(self.server_addr) - We'd use it like so: x = Client(192.168.0.1) import pickle pickle.dump(x) #getstate gets called and returns IP for pickling. #program exits, we restart it x=Client(None) x = pickle

Re: Questions on Pickle and Shelve

2015-11-06 Thread Peter Otten
Virgil Stokes wrote: > Here is snippet of Python (vers. 2.7.10) code that bothers me. > > import cPickle as pickle > > print "Pickle lists:" > dogs = ['Rover','King','Spot','Rufus'] > cats = ['Mimi','Misty

Re: Questions on Pickle and Shelve

2015-11-06 Thread Chris Warrick
On 6 November 2015 at 12:53, Virgil Stokes wrote: > Here is snippet of Python (vers. 2.7.10) code that bothers me. > > [snip bad code] > > 1) Which (the pickle or shelve code) takes less total RAM, if dogs and cats > were very large? > 2) When the last shelve.open is

Questions on Pickle and Shelve

2015-11-06 Thread Virgil Stokes
Here is snippet of Python (vers. 2.7.10) code that bothers me. import cPickle as pickle print "Pickle lists:" dogs = ['Rover','King','Spot','Rufus'] cats = ['Mimi','Misty','Sasha'] with open('pickle.dat'

Re: enhancement request: make py3 read/write py2 pickle format

2015-06-11 Thread Serhiy Storchaka
On 11.06.15 02:58, Chris Angelico wrote: On Thu, Jun 11, 2015 at 8:10 AM, Devin Jeanpierre wrote: The problem is that there are two different ways repr might write out a dict equal to {'a': 1, 'b': 2}. This can make tests brittle -- e.g. it's why doctest fails badly at examples involving dictio

Re: enhancement request: make py3 read/write py2 pickle format

2015-06-11 Thread Steven D'Aprano
le to > (but smaller) than the set of types you get from literal_eval/repr. Well, what do people do when they want to serialise something like MyClass, but have to use (say) JSON rather than pickle? I'd write a method to export enough information (as JSON) to reconstruct the instance,

Re: enhancement request: make py3 read/write py2 pickle format

2015-06-10 Thread Devin Jeanpierre
Snipped aplenty. On Wed, Jun 10, 2015 at 8:21 PM, Steven D'Aprano wrote: > On Thu, 11 Jun 2015 08:10 am, Devin Jeanpierre wrote: > [...] >> I could spend a bunch of time writing yet another config file format, >> or I could use text format protocol buffers, YAML, or TOML and call it >> a day. > >

  1   2   3   4   5   6   7   8   9   >