John Nagle wrote: > I'm starting to think that the "cpickle" module, which Python 3 > uses by default, has a problem. After the program has been > running for a while, I start seeing errors such as > > File "C:\projects\sitetruth\InfoSiteRating.py", line 200, in scansite > if len(self.badbusinessinfo) > 0 : # if bad stuff > NameError: name 'len' is not defined > > which ought to be impossible in Python, and
"Impossible"? py> len <built-in function len> py> import __builtin__ # use builtins in Python 3 py> del __builtin__.len py> len Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'len' is not defined Why something is deleting builtins len is a mystery. Sounds to me that your Python installation is borked. > File "C:\projects\sitetruth\subprocesscall.py", line 129, in send > self.writer.dump(args) # send data > OSError: [Errno 22] Invalid argument > > from somewhere deep inside CPickle. Why do you say "deep inside CPickle"? The traceback says C:\projects\sitetruth\subprocesscall.py Is it possible you have accidentally shadowed the CPickle module with something? What does this say? import cPickle print cPickle.__file__ Use _pickle in Python 3. > I got > > File "C:\projects\sitetruth\InfoSiteRating.py", line 223, in > get_rating_text > (ratingsmalliconurl, ratinglargiconurl, ratingalttext) = > DetailsPageBuilder.getratingiconinfo(rating) > NameError: name 'DetailsPageBuilder' is not defined > (That's an imported module. It worked earlier in the run.) > > and finally, even after I deleted all .pyc files and all Python > cache directories: > > Fatal Python error: GC object already tracked > > Current thread 0x00001a14 (most recent call first): > File "C:\python34\lib\site-packages\pymysql\connections.py", line 411 > in description > File "C:\python34\lib\site-packages\pymysql\connections.py", line 1248 > in _get_descriptions > File "C:\python34\lib\site-packages\pymysql\connections.py", line 1182 > in _read_result_packet > File "C:\python34\lib\site-packages\pymysql\connections.py", line 1132 > in read > File "C:\python34\lib\site-packages\pymysql\connections.py", line 929 > in _read_query_result > File "C:\python34\lib\site-packages\pymysql\connections.py", line 768 > in query > File "C:\python34\lib\site-packages\pymysql\cursors.py", line 282 in > _query > File "C:\python34\lib\site-packages\pymysql\cursors.py", line 134 in > execute > File "C:\projects\sitetruth\domaincacheitem.py", line 128 in select > File "C:\projects\sitetruth\domaincache.py", line 30 in search > File "C:\projects\sitetruth\ratesite.py", line 31 in ratedomain > File "C:\projects\sitetruth\RatingProcess.py", line 68 in call > File "C:\projects\sitetruth\subprocesscall.py", line 140 in docall > File "C:\projects\sitetruth\subprocesscall.py", line 158 in run > File "C:\projects\sitetruth\RatingProcess.py", line 89 in main > File "C:\projects\sitetruth\RatingProcess.py", line 95 in <module> > > That's a definite memory error. > > So something is corrupting memory. Probably CPickle. > All my code is in Python. Every library module came in via "pip", into a > clean Python 3.4.3 (32 bit) installation on Win7/x86-64. > Currently installed packages: > > beautifulsoup4 (4.3.2) > dnspython3 (1.12.0) > html5lib (0.999) > pip (6.0.8) > PyMySQL (0.6.6) > pyparsing (2.0.3) > setuptools (12.0.5) > six (1.9.0) > > And it works fine with Python 2.7.9. > > Is there some way to force the use of the pure Python pickle module? Try renaming the _pickle module. This works on Linux: mv /usr/local/lib/python3.3/lib-dynload/_pickle.cpython-33m.so /usr/local/lib/python3.3/lib-dynload/_pickle.cpython-33m.so~ > My guess is that there's something about reusing "pickle" instances > that botches memory uses in CPython 3's C code for "cpickle". How are you reusing instances? -- Steven -- https://mail.python.org/mailman/listinfo/python-list
