Error with co_filename when loading modules from zip file
Hi,
I'm using a program that distributes python in a zip
file and ran into an issue with the logging package.
It seems to return the wrong filename/line number when
loading python from a zip file. Please help!
I'm using python31, and have copied the lib directory to
/home/user/python3.1
and have created a zip of that directory and placed it in
/home/user/python3.1/python31.zip
The logging package gets the filename and line number
of the calling function by looking at two variables, the filename
of the frame in the stack trace and the variable logging._srcfile.
The comparison is done in logging/__init__.py:findCaller.
In the situation above, when I run,
PYTHONPATH=/home/user/python3.1 ./myexe run.py
I see
filename=/home/user/python3.1/logging/__init__.py
_srcfile=/home/user/python3.1/logging/__init__.py
Here, filename and _srcfile are the same, so the logger correctly
outputs the filename of run.py.
When I run,
PYTHONPATH=/home/user/python3.1/python31.zip ./myexe run.py
I see
filename=/home/user/python3.1/logging/__init__.py
_srcfile=/home/user/python3.1/python31.zip/logging/__init__.py
Here, filename and _srcfile are different, so the logger incorrectly
outputs the filename of /home/user/python3.1/logging/__init__.py
I've noticed this:
- the filename seems to be set when you compile the module
- it seems to be set when you load the module (even after moving it)
- it does not seem to get set when you load the module from
the pyc in the zip file!
I've tried putting only the pyc files, only the py files
and both in the zip file.
Any help?
Thanks,
Bob
run.py:
import logging
class Handler(logging.Handler):
def __init__(self):
logging.Handler.__init__(self)
def emit(self, record):
print('message: ' + record.msg)
print('filename: ' + record.pathname)
print('line: ' + str(record.lineno))
logging.getLogger().addHandler(Handler())
logging.error('hi')
--
http://mail.python.org/mailman/listinfo/python-list
consecutive node sequence and pathlength problem using networkx graph
Let say,I have a conjugated cyclic polygon and its nodes are given by the list: list_p=[a,b,c,d,e,f,g,a,a,b,d,d,d,d,d,c,c,e,e,a,d,d,g]. If X & Y are any elements in a list_p except d, and Z is also an element of list_p but has value only d, i.e, Z=d. Now,I want to compute the number of paths with sequence X-Z- Y consecutive nodes, example: a-d-d-e,a-d-d-a, a-d-d-d-g, etc. Consecutive path length of Z can be 1,2,3,4 or 5. To generalize: Find total number of paths with consecutive X-Z-Y sequence for path length of z within range(1,6): or (i) X-Z-Y for len(Z)=1 (ii) X-Z-Y for len(Z)=2 (iii)X-Z-Y for len(Z)=3 (iv) X-Z-Y for len(Z)=4 (v) X-Z-Y for len(Z)=5 Is there any easy way to program this using python networkx or igraph? -- http://mail.python.org/mailman/listinfo/python-list
RE: ModuleNotFoundError: No module named 'email.mime'; 'email' is not a package
Just registered
Thanks
Original Message
Subject: ModuleNotFoundError: No module named 'email.mime'; 'email' is
not a package
From: <[1]b...@bbhoyer.com>
Date: Sun, December 08, 2019 11:14 am
To: [2]python-list@python.org
Hello Python Team,
I am a beginner in Python, been working on class material from Mosh
youtube 6 hrs., Mosh 12 hr paid course; Python Cash Course by Eric
Matthes … most of the time everything works just fine or a little
adjustment for updated versions.
I am running Python 3.8.0
In example, Mosh 12 hr course, he did an exercise on sending email from
within Python, so here is the code :
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
import smtplib
message = MIMEMultipart()
message["from"] = "Bob Hoyer = Python"
message["to"] = "[3]b...@bbhoyer.com"
message["subject"] = "This is a test using Python"
message.attach(MIMEText("Body"))
# godaddy recommended SMTP_SSL, host name & port #
with smtplib.SMTP(host="smtpout.secureserver.net", port=465) as smtp:
smtp.ehlo()
# godaddy recommended removing ...
# smtp.starttls()
smtp.login("[4]em...@email.com", "password") #email login &
password blanked out for this msg
smtp.send_message(message)
print("Sent ...")
smtp.close()
Here is the error message:
Traceback (most recent call last):
File "c:\Users\Owner\Desktop\HelloWorld\[5]email.py", line 1, in
from email.mime.multipart import MIMEMultipart
File "c:\Users\Owner\Desktop\HelloWorld\[6]email.py", line 1, in
from email.mime.multipart import MIMEMultipart
ModuleNotFoundError: No module named 'email.mime'; 'email' is not a package
I have spent some time trying to figure out resolve ...
Can you help me with this pistol of a problem …
Bob Hoyer
References
Visible links
1. mailto:b...@bbhoyer.com
2. mailto:python-list@python.org
3. mailto:b...@bbhoyer.com
4. mailto:em...@email.com
5. http://email.py/
6. http://email.py/
--
https://mail.python.org/mailman/listinfo/python-list
Accessing Outlook Public Folders
Hi,
I'm new to Python and I want to create a script that will import
contactitems into an Outlook Public Folder.
Therefore I've installed the Python for Windows Extensions.
This way I can access my personal contacts
import win32com.client
OutlookObj = win32com.client.Dispatch("Outlook.Application")
Nms = OutlookObj.GetNameSpace("MAPI")
# Personal contacts folder
custs = Nms.GetDefaultFolder(10).Items
print custs.count
for i in range(custs.count):
cust = custs[i]
print cust.FullName + " -> " + cust.FileAs
But when I try to access contacts in the public folder 'klanten' with
this code I get an error.
import win32com.client
OutlookObj = win32com.client.Dispatch("Outlook.Application")
Nms = OutlookObj.GetNameSpace("MAPI")
# Folder Klanten from public folders
custs = Nms.Folders.Item("Public Folders").Folders.Item("All Public
Folders").Folders["Klanten"].Items
print custs.count
for i in range(custs.count):
cust = custs[i]
print cust.FullName + " -> " + cust.FileAs
Traceback (most recent call last):
File "D:\PythonScripts\tp2outlook.py", line 13, in -toplevel-
print cust.FullName + " -> " + cust.FileAs
File "D:\Python23\Lib\site-packages\win32com\client\dynamic.py", line
489, in __getattr__
raise AttributeError, "%s.%s" % (self._username_, attr)
AttributeError: .FullName
Anyone an idea how I can access those contactitems?
Thanks,
Bob
--
http://mail.python.org/mailman/listinfo/python-list
pulling info from website
i am trying to write a script for Xbox media center that will pull information from the bbc news website and display the headlines , how do i pull this info into a list??? -- http://mail.python.org/mailman/listinfo/python-list
Re: pySerial Windows write problem
I forgot to mention that once the Python program(s) fail, THEN the C++ program also fails to opne the port, and the equipment has to be power-cycled to get things to work again. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: pySerial Windows write problem
Thanks heaps for the help, guys! I bit the bullit and am now leaving the serial ports open all of the time. I still think it is a driver/hardware problem. When the ports go South even a reboot of the laptop the program is running on won't fix it. The hardware is all brand new, so it should have some bugs in it. Shouldn't it? :) Anyway, here's a short page about the program, just for fun www.greschke.com/unlinked/pocus01.htm Thanks! Bob -- http://mail.python.org/mailman/listinfo/python-list
problem with datetime
Relatively new to python. I can get the following to work from the command line: Python 2.3.4 (#2, Aug 18 2004, 21:49:15) [GCC 3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import datetime >>> d = datetime.datetime.today() >>> d datetime.datetime(2004, 12, 10, 6, 13, 28, 154472) >>> But when I try to run the following small program I get the following results: import datetime d = datetime.datetime.today() print d Traceback (most recent call last): File "datetime.py", line 1, in ? import datetime File "/home/bob/pyshow/datetime.py", line 3, in ? d = datetime.datetime.today() AttributeError: 'module' object has no attribute 'today' Do I need to change a path? Running python 2.3.4 on linux redhat 7.3. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with datetime
Thanks. That did it. And I know better than to use the module name. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: 70% [* SPAM *] Re: multiprocessing.Queue blocks when sending large object
On Mon, 5 Dec 2011 09:02:08 +0100, DPalao wrote: >El Martes Noviembre 29 2011, DPalao escribió: >> Hello, >> I'm trying to use multiprocessing to parallelize a code. There is a number >> of tasks (usually 12) that can be run independently. Each task produces a >> numpy array, and at the end, those arrays must be combined. >> I implemented this using Queues (multiprocessing.Queue): one for input and >> another for output. >> But the code blocks. And it must be related to the size of the item I put >> on the Queue: if I put a small array, the code works well; if the array is >> realistically large (in my case if can vary from 160kB to 1MB), the code >> blocks apparently forever. >> I have tried this: >> http://www.bryceboe.com/2011/01/28/the-python-multiprocessing-queue-and-lar >> ge- objects/ >> but it didn't work (especifically I put a None sentinel at the end for each >> worker). >> >> Before I change the implementation, >> is there a way to bypass this problem with multiprocessing.Queue? >> Should I post the code (or a sketchy version of it)? >> >> TIA, >> >> David > >Just for reference. The other day I found the explanation by "ryles" on >his/her mail of 27th aug 2009, with title "Re: Q: multiprocessing.Queue size >limitations or bug...". It is very clarifying. >After having read that I arranged the program such that the main process did >not need to know when the others have finished, so I changed the process join >call with a queue get call, until a None (one per process) is returned. > >Best, > >David Why do people add character like[* SPAM *] to their subject lines ?? Is it supposed to do something ?? I figured since programmers hang out here, maybe one of you know this. Thanks, boB -- http://mail.python.org/mailman/listinfo/python-list
python and dbus - beginner seeking some help
Hello, First of all, does anyone know whether dbus will allow be to do 3 things: 1. query whether or not there is a dvd drive. 2. query whether or not there is a blank disc in it. 3. query what it's capacity is. If I'm barking up the wrong tree, what should I be considering instead ? If I am thinking along the right lines, then I'd be grateful for any links to any guides or tutorials relating to dbus. I have tried looking and found nothing uesful yet, and I will continue to do so, but I was hoping someone could help. Thanks for reading :-) -- http://mail.python.org/mailman/listinfo/python-list
python and dbus - beginner seeking some help
I've found some information at: http://www.freedesktop.org/wiki/Software/dbus However, if anyone knows of simpler or perhaps just easier to follow articles online or knows that I'm barking up the wrong tree, your help will still be appriciated. Meanwhile I'm off to RTFM to see if I can make sense of it or need more help :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: python and dbus - beginner seeking some help
On Fri, 28 Jul 2006 19:03:00 +0100, bob wrote: > Hello, > > First of all, does anyone know whether dbus will allow be to do 3 things: > > 1. query whether or not there is a dvd drive. > 2. query whether or not there is a blank disc in it. > 3. query what it's capacity is. > > If I'm barking up the wrong tree, what should I be considering instead ? > > If I am thinking along the right lines, then I'd be grateful for any links > to any guides or tutorials relating to dbus. I have tried looking and > found nothing uesful yet, and I will continue to do so, but I was hoping > someone could help. > > Thanks for reading :-) I've found some info at: http://www.freedesktop.org/wiki/Software/dbus but even still, additional info can't do any harm, especially if I struggle to make sense of the information at the above location. -- http://mail.python.org/mailman/listinfo/python-list
simple dbus python problem ... please help
Hi,
I have this sample python script from the hal sources, but it doesn't work
for me. This is despite other example python scripts I have to help me are
working fine. The problem is that this script is the closet to what it is
I actually want to learn to do.
The error is:
Traceback (most recent call last):
File "./dbus-3.py", line 6, in ?
hal_service = bus.get_service ('org.freedesktop.Hal')
AttributeError: 'SystemBus' object has no attribute 'get_service'
The script is as follows:
#!/usr/bin/python
import dbus
bus = dbus.Bus (dbus.Bus.TYPE_SYSTEM)
hal_service = bus.get_service ('org.freedesktop.Hal')
hal_manager = hal_service.get_object ('/org/freedesktop/Hal/Manager',
'org.freedesktop.Hal.Manager')
volume_udi_list = hal_manager.FindDeviceByCapability ('volume')
for udi in volume_udi_list:
volume = hal_service.get_object (udi, 'org.freedesktop.Hal.Device')
device_file = volume.GetProperty ('block.device')
fstype = volume.GetProperty ('volume.fstype')
storage_udi = volume.GetProperty ('block.storage_device')
storage = hal_service.get_object (storage_udi, 'org.freedesktop.Hal.Device')
drive_type = storage.GetProperty ('storage.drive_type')
print 'udi=%s device_file=%s fstype=%s drive_type=%s'%(udi,
device_file, fstype, drive_type)
Documentation on this stuff is extremely thin on the ground, so links to
any good resources other than those at freedesktop.org would be most
appreciated!
All I really want to do is have a nice routine to do the following:
1. check whether or not a user has a dvd drive.
2. check whether a disc is in it.
3. if there is a disc in the drive, check the various properties relating
to it.
Surely it cannot be that hard to get enough information to work out how to
do that!
Thanks for reading :)
--
http://mail.python.org/mailman/listinfo/python-list
How to move optparse from main to function?
I'm playing around with optparse and created the code below. How do I
move that to a function and what variable do I pass?
>From the documentation it seems like "options.optparse_test" would have
the value zero if its option, either "-t" or "--test", is not detected
at the command line. When I issue "print options.optparse_test" when no
command options are used, "None" is sent to the screen (IIRC)... Is a
proper check if a command line argument is not used to use the
following:
# being check
if options.optparse_test <> 1:
print "test parameter NOT detected"
# end check
Thanks.
# begin program
#!/usr/bin/python
from optparse import OptionParser
import string
parser = OptionParser()
parser.add_option("-t", "--test", action="count", dest="optparse_test",
help="testing optparse")
(options, args) = parser.parse_args()
print options.optparse_test
if options.optparse_test == 1:
print "test parameter detected"
if options.optparse_test <> 1:
print "test parameter NOT detected"
#parser.print_help()
# end program
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to move optparse from main to function?
The module documentation helped me construct the meat of my code bu it didn't lend a hand on how to build the real meal deal the way Jason's explanation did. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to move optparse from main to function?
Yes the documentation is helpful, but I wouldn't have been able to do what you did in your code by just looking at section 6.21.2.9. I thought I could put "parser = parserSetup()" and "(options, args) = parser.parse_args()" in the function. Thanks for helping out with that! -- http://mail.python.org/mailman/listinfo/python-list
How would you open this file?
I want to open a file in a function and pass the file back to main so another function can manipulate the file, something like this: # begin def open_file() filename=open(options.filename_input,'r') return filename open_file() print filename.read() filename.close() # end But this doesn't work... It appears that "open_file()" needs a variable, but in order to pass a variable to "open_file()" I first need a variable... but I would rather have the function send me "filename" without creating it main. Is that possible? Is there a better way to have a function open a file and pass the file back to main than what I am trying to do? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: How would you open this file?
Thanks all! That helps out a lot! Python is kinda' cool... -- http://mail.python.org/mailman/listinfo/python-list
Can optparse do dependencies?
I'd like to setup command line switches that are dependent on other
switches, similar to what rpm does listed below. From the grammar below
we see that the "query-options" are dependent on the query switch,
{-q|--query}. Can "optparse" do this or do I have to code my own
"thing"? Thanks.
QUERYING AND VERIFYING PACKAGES:
rpm {-q|--query} [select-options] [query-options]
...
query-options
[--changelog] [-c,--configfiles] [-d,--docfiles] [--dump]
[--filesbypkg] [-i,--info] [--last] [-l,--list]
[--provides] [--qf,--queryformat QUERYFMT]
[-R,--requires] [--scripts] [-s,--state]
[--triggers,--triggerscripts]
--
http://mail.python.org/mailman/listinfo/python-list
trying to gzip uncompress a StringIO
I'm using the code below to read the zipped, base64 encoded WMF file saved in an XML file with "Save as XML" from MS Word. As the "At this point" comment shows, I know that the base64 decoding is going fine, but unzipping from the decodedVersion StringIO object isn't getting me anything, because the len(fileContent) call is returning 0. Any suggestions? thanks, Bob ### import sys, base64, gzip, StringIO infile = sys.argv[1] outfile = sys.argv[2] # When unencoding base64 file, write to a string that acts like a file decodedVersion = StringIO.StringIO() base64.decode(open(infile, 'r'),decodedVersion) # At this point, if we write out decodedVersion.getvalue() to a file, # gzip -d makes a proper wmf file from it, so we know everything's OK so far. # following based on # http://www.diveintopython.org/http_web_services/gzip_compression.html fileObj = gzip.GzipFile(fileobj=decodedVersion); fileContent = fileObj.read() print len(fileContent) -- http://mail.python.org/mailman/listinfo/python-list
Re: trying to gzip uncompress a StringIO
Perfect, thanks! Now I have a working WMF file and everything. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: trying to gzip uncompress a StringIO
Perfect, thanks! Now I have a working WMF file and everything. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: trying to gzip uncompress a StringIO
Perfect, thanks! Now I have a working WMF file and everything. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: trying to gzip uncompress a StringIO
Perfect, thanks! Now I have a working WMF file and everything. Bob -- http://mail.python.org/mailman/listinfo/python-list
iputils module
Anyone know where to get the iputils module for python? -- http://mail.python.org/mailman/listinfo/python-list
regex to exctract informations
Dears,
I am trying to search and replace strings with regex.
The string is identified by a keyword :
IDImage("1M234567");
DescriptionImage("Desc of the Image 1");
I want to exctract the IDImage (1M234567 ) and the Description.
The ID are characters and numbers, the description too.
Thx,
Bertrand
--
http://mail.python.org/mailman/listinfo/python-list
Cannot catch _mysql_exceptions.OperationalError
In our database code (we are using django v0.96) we wanted to catch
and handle MySQL OperationalErrors. We use both the Django models and
database connections.
However, even though we confirmed that a
_mysql_exceptions.OperationalError are raised (particularly 1213
deadlock), we cannot catch them with try/except.
Here's the code that did not work:
import _mysql_exceptions
from _mysql_exceptions import OperationalError
try:
database_code()
except (_mysql_exceptions.OperationalError, OperationalError), e:
error_handling()
Originally, we just had one import, but tried both to ensure that was
not the problem. In debugging, we confirmed that both type(e) and
e.__class_ were .
The only work-around we found was:
try:
database_code()
except Exception, e:
if e.__class__.__name__.find('OperationalError') != -1:
error_handling()
else:
raise
This seems silly. If we used any other exception, the except clause
seems to work correctly.
Is there a problem with _mysql_exceptions.OperationalError? Is there
a better work-around?
Thanks,
Bob
--
http://mail.python.org/mailman/listinfo/python-list
messenger plus! live
Messenger Plus! Live is an add-on for Windows Live Messenger that adds tons of features and extras to the software. Extend the possibilities of Messenger and make your experience a lot more entertaining! For more on this subject check out this cool site www.msgpluslive.net -- http://mail.python.org/mailman/listinfo/python-list
can't send email
Hello.
I'm trying to send email using python 2.6.1 under snow leopard, but I
can't get it to work. I'm trying one of the many examples I found on
the web
EXAMPLE 1
import smtplib
fromaddr = 'fromu...@gmail.com'
toaddrs = 'tou...@gmail.com'
msg = 'There was a terrible error that occured and I wanted you to
know!'
# Credentials (if needed)
username = 'username'
password = 'password'
# The actual mail send
server = smtplib.SMTP('smtp.gmail.com:587')
server.starttls()
server.login(username,password)
server.sendmail(fromaddr, toaddrs, msg)
server.quit()
EXAMPLE 2
# Import smtplib for the actual sending function
import smtplib
# Import the email modules we'll need
from email.mime.text import MIMEText
# Open a plain text file for reading. For this example, assume that
# the text file contains only ASCII characters.
fp = open(textfile, 'rb')
# Create a text/plain message
msg = MIMEText(fp.read())
fp.close()
# me == the sender's email address
# you == the recipient's email address
msg['Subject'] = 'The contents of %s' % textfile
msg['From'] = me
msg['To'] = you
# Send the message via our own SMTP server, but don't include the
# envelope header.
s = smtplib.SMTP()
s.sendmail(me, [you], msg.as_string())
s.quit()
The error I get is this
python email.py
Traceback (most recent call last):
File "email.py", line 2, in
import smtplib
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/smtplib.py", line 46, in
import email.utils
File "/Users/Bob/Code/email.py", line 5, in
from email.mime.text import MIMEText
ImportError: No module named mime.text
I'm using the native python version installed by apple, so I don't
know why email and email.utils and mime are not found. I checked on
the filesystem and they are present...
Thanks
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't send email
On Sep 7, 10:27 pm, Chris Rebert wrote: > On Tue, Sep 7, 2010 at 1:12 PM, Bob wrote: > > Hello. > > I'm trying to send email using python 2.6.1 under snow leopard, but I > > can't get it to work. I'm trying one of the many examples I found on > > the web > > > The error I get is this > > > python email.py > > Traceback (most recent call last): > > File "email.py", line 2, in > > import smtplib > > File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/ > > python2.6/smtplib.py", line 46, in > > import email.utils > > File "/Users/Bob/Code/email.py", line 5, in > > from email.mime.text import MIMEText > > ImportError: No module named mime.text > > > I'm using the native python version installed by apple, so I don't > > know why email and email.utils and mime are not found. I checked on > > the filesystem and they are present... > > Your /Users/Bob/Code/email.py file is shadowing the std lib "email" > module. Rename your file to something else so that it no longer > conflicts. > > Cheers, > Chris > --http://blog.rebertia.com yes I just realized that... it works now!! thanks -- http://mail.python.org/mailman/listinfo/python-list
formatted input
Hi All, I have another question about formatted input. Suppose I am reading a text file, and that I want it to be something like this word11 = num11, word12 = num12, word13 = num13 etc... word21 = num21, word22 = num12, word23 = num23 etc... etc... where wordx1 belongs to a certain dictionary of words, say dic1, while wordx2 belongs to dic2, the numbers within some range and so on. I was wondering if there is something in the standard library I may use to check whether the file I am reading has a correct syntax according to my rules/dictionaries instead of implementing my own routine that would look like (pseudocode) for each line put words into a list check condition for each word Thanks -- http://mail.python.org/mailman/listinfo/python-list
uninitialize PyEval_InitThreads()
Hi, I see that I can initialize python for multi thread support by calling PyEval_InitThreads(). I'm under the impression that when the GIL is turned on, python can slow down a measurable amount. Since I only need to use multiple threads for a portion of my program, after I call PyEval_InitThreads() and use the threads, is there a way to disable multi threading support in python? (I noticed that _PyEval_FiniThreads was undocumented and presumably shouldn't be called) I would only want to do this if it would increase run time performance in the main thread. So if that assumption is wrong, please let me know. Thanks, Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: distutils bdist_wininst failure on Linux
Steven D'Aprano-11 wrote
>
> And I have a work-around that seems to work for me. Put this at the top
> of your setup.py install script:
>
>
>
> # Work around mbcs bug in distutils.
> # http://bugs.python.org/issue10945
> import codecs
> try:
> codecs.lookup('mbcs')
> except LookupError:
> ascii = codecs.lookup('ascii')
> func = lambda name, enc=ascii: {True: enc}.get(name=='mbcs')
> codecs.register(func)
>
>
Nice one, worked first time! Thanks!
--
View this message in context:
http://python.6.n6.nabble.com/distutils-bdist-wininst-failure-on-Linux-tp4498729p4984209.html
Sent from the Python - python-list mailing list archive at Nabble.com.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Top-posting &c. (was Re: [ANNC] pybotwar-0.8)
in 679182 20120821 181439 Dennis Lee Bieber wrote: >On Tue, 21 Aug 2012 08:07:33 +0200, Alex Strickland >declaimed the following in gmane.comp.python.general: > >> On 2012/08/17 12:42 AM, Madison May wrote: >> >> > As a lurker, I agree completely with Chris's sentiments. >> >> I too, but I'd prefer something top-posted than have to skip through 38 >> pages of quoted e-mail to get to a (generally) 1 liner at the bottom. > >Doesn't help me though... Agent shows quoted material as blue, fresh >text as black. > >I tend to not see a one-liner at the top (since it is next to the >attribution line) and if the rest of the page is all blue text I hit >page down... and down, down, down... looking for black text... Then end >up going "Wha', where's the new stuff?" and having to scroll back up to >find a one-liner cuddling up with the attribution line. Yep, and the only solution is for everyone to top-post. -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie: where's the new python gone?
c.c Description: Binary data Op 10 Sep 2012, om 22:53 heeft William R. Wing (Bill Wing) het volgende geschreven: > On Sep 10, 2012, at 11:17 AM, Bob Aalsma wrote: > >> Well, Bill, better late than never - thanks for stepping in. >> You are right, my problems are not yet solved ;) > > As Hans pointed out, you are looking for python, not Python (the frameworks > are named Python, the executable is python). Sorry about that. > > To eliminate the output from TimeMachine (assuming it is using an external > drive or a TimeCapsule), add one more option to the find command: > > $ sudo find -x / -name python -print (or follow his directions for piping to > a file). > > The -x option prevents find from descending into a volume with a different > device number from the one you started on. > > -Bill > -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie: where's the new python gone?
Hmm, this feels embarrassing but the good news is that, on seeing the errors, I remember using a "sudo" with the make install and only later finding out that I shouldn't have. Last login: Tue Sep 11 09:46:11 on ttys000 macpro1:~ debaas$ pwd /Users/debaas macpro1:~ debaas$ ls Desktop Documents Downloads Library Movies Music PicturesPublic Sites macpro1:~ debaas$ cd Desktop/ macpro1:Desktop debaas$ cd Python-2.7.3/ macpro1:Python-2.7.3 debaas$ ls /usr/local apache2 bin clamXav include lib libexec share macpro1:Python-2.7.3 debaas$ make install /usr/bin/install -c python.exe /usr/local/bin/python2.7 install: /usr/local/bin/python2.7: Permission denied make: *** [altbininstall] Error 71 So how to proceed here? -Bob Op 11 Sep 2012, om 14:26 heeft William R. Wing (Bill Wing) het volgende geschreven: > On Sep 11, 2012, at 3:52 AM, Bob Aalsma wrote: > >> >> Op 10 Sep 2012, om 22:53 heeft William R. Wing (Bill Wing) het volgende >> geschreven: >> >>> On Sep 10, 2012, at 11:17 AM, Bob Aalsma >>> wrote: >>> >>>> Well, Bill, better late than never - thanks for stepping in. >>>> You are right, my problems are not yet solved ;) >>> >>> As Hans pointed out, you are looking for python, not Python (the frameworks >>> are named Python, the executable is python). Sorry about that. >>> >>> To eliminate the output from TimeMachine (assuming it is using an external >>> drive or a TimeCapsule), add one more option to the find command: >>> >>> $ sudo find -x / -name python -print (or follow his directions for piping >>> to a file). >>> >>> The -x option prevents find from descending into a volume with a different >>> device number from the one you started on. >>> >>> -Bill >>> >> >> -- >> http://mail.python.org/mailman/listinfo/python-list > > Well, clearly, the only copies of python 2.7 on your system are from Apple > (in System/Library and associated with Xcode). > So, either the one you downloaded overwrote one of them, or the install > failed and you didn't notice the error go zooming by in a listing. > > Can you run the install procedure again and capture the output? > > -Bill -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie: where's the new python gone?
On 11 Sep 2012, at 15:31, William R. Wing (Bill Wing) wrote: > On Sep 11, 2012, at 9:12 AM, Bob Aalsma wrote: > >> Hmm, this feels embarrassing but the good news is that, on seeing the >> errors, I remember using a "sudo" with the make install and only later >> finding out that I shouldn't have. >> >> Last login: Tue Sep 11 09:46:11 on ttys000 >> macpro1:~ debaas$ pwd >> /Users/debaas >> macpro1:~ debaas$ ls >> Desktop Documents Downloads Library Movies >> Music PicturesPublic Sites >> macpro1:~ debaas$ cd Desktop/ >> macpro1:Desktop debaas$ cd Python-2.7.3/ >> macpro1:Python-2.7.3 debaas$ ls /usr/local >> apache2 bin clamXav include lib libexec share >> macpro1:Python-2.7.3 debaas$ make install >> /usr/bin/install -c python.exe /usr/local/bin/python2.7 >> install: /usr/local/bin/python2.7: Permission denied >> make: *** [altbininstall] Error 71 >> >> So how to proceed here? >> >> -Bob > > As others have commented, please don't top-post. > > The script from which you've captured a fragment of output above isn't going > to work on a Mac without rewriting. (Your Mac isn't going to like python.exe > for example.) > > As to proceeding, in all honesty, the simplest thing to do is go here: > > http://www.python.org/download/ > > Look down the page to the link for Python 2.7.3 Mac OS X 64-bit/32-bit > x86-64/i386 Installer (for Mac OS X 10.6 and 10.7 [2]) and grab it. That > will hand you a completely prebuilt 2.7.3 (latest) along with an installer > that will put it in /Library/Frameworks where it belongs and will even edit > your .profile. > > Good luck, > Bill Sorry about top-posting. And it worked! Thank you for teaching! Regards, Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Article on the future of Python
in 681910 20120927 131113 Devin Jeanpierre wrote: >On Thu, Sep 27, 2012 at 2:13 AM, Steven D'Aprano > wrote: >> On Tue, 25 Sep 2012 09:15:00 +0100, Mark Lawrence wrote: >> And a response: >> >> http://data.geek.nz/python-is-doing-just-fine > >Summary of that article: > >"Sure, you have all these legitimate concerns, but look, cake!" Quote : "This piece argues that Python is an easy-to-learn language that where you can be almost immediately productive in." -- http://mail.python.org/mailman/listinfo/python-list
Re: RE: Unpaking Tuple
in 682592 20121008 232126 "Prasad, Ramit" wrote: >Thomas Bach wrote:=0D=0A> Hi there,=0D=0A> =0D=0A> On Sat, Oct 06, 2012 at = >03:08:38PM +, Steven D'Aprano wrote:=0D=0A> >=0D=0A> > my_tuple =3D my_= >tuple[:4]=0D=0A> > a,b,c,d =3D my_tuple if len(my_tuple) =3D=3D 4 else (my_= >tuple + (None,)*4)[:4]=0D=0A> >=0D=0A> =0D=0A> Are you sure this works as y= >ou expect? I just stumbled over the following:=0D=0A> =0D=0A> $ python=0D= >=0A> Python 3=2E2=2E3 (default, Jun 25 2012, 23:10:56)=0D=0A> [GCC 4=2E7=2E= >1] on linux2=0D=0A> Type "help", "copyright", "credits" or "license" for mo= >re information=2E=0D=0A> >>> split =3D ['foo', 'bar']=0D=0A> >>> head, tail= >=3D split if len(split) =3D=3D 2 else split[0], None=0D=0A> >>> head=0D=0A= >> ['foo', 'bar']=0D=0A> >>> tail=0D=0A> >>>=0D=0A> =0D=0A> I don't get it! = >Could someone help me, please? Why is head not 'foo'=0D=0A> and tail not 'b= >ar'?=0D=0A> =0D=0A> Regards,=0D=0A>Thomas=0D=0A> --=0D=0A=0D=0AI think yo= >u just need to wrap the else in parenthesis so the=0D=0Aelse clause is trea= >ted as a tuple=2E Without the parenthesis =0D=0AI believe it is grouping th= >e code like this=2E=0D=0A=0D=0Ahead, tail =3D (split if len(split) =3D=3D 2= >else split[0] ), None=0D=0A=0D=0AYou want:=0D=0Ahead, tail =3D split if le= >n(split) =3D=3D 2 else (split[0], None )=0D=0A=0D=0A=0D=0ARamit=0D=0AThis e= >mail is confidential and subject to important disclaimers and=0D=0Aconditio= >ns including on offers for the purchase or sale of=0D=0Asecurities, accurac= >y and completeness of information, viruses,=0D=0Aconfidentiality, legal pri= >vilege, and legal entity disclaimers,=0D=0Aavailable at http://www=2Ejpmorg= >an=2Ecom/pages/disclosures/email=2E How does one unpack this post? ;-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Obnoxious postings from Google Groups
in 684220 20121102 093654 Jamie Paul Griffin wrote: >/ ru...@yahoo.com wrote on Thu 1.Nov'12 at 15:08:26 -0700 / > >> On 11/01/2012 03:55 AM, Jamie Paul Griffin wrote: >> > Anybody serious about programming should be using a form of >> > UNIX/Linux if you ask me. It's inconceivable that these systems >> > should be avoided if you're serious about Software Engineering and >> > Computer Science, etc. For UNIX there are loads of decent news >> > reading software and mail user agents to learn and use. slrn is a >> > good one and point it at gmane.org as someone else pointed out. I >> > can't even imagine using a browser or Google Groups, etc. now. > >> Are you saying that this group is only for "serious" programmers? > >I don't see where my comments suggested that this group is only for serious >programmers. I simply believe that the UNIX platform, in whatever form, is >better placed and designed for all sorts of computing and engineering >projects. The history of UNIX speaks for itself. Many Universities that offer >respected and credible science based degree programmes, namely engineering and >computing programmes, strongly encourage students to become competent with >UNIX systems. Windows in my opinion is really for those who use the internet >on a casual basis or in a commercial environment where its staff are not >necessarily computer literate and therefore need a platform that they can use >which doesn't require them to learn more complex techniques and protocols. >But, having said that, I'm not against Windows at all. I use it frequently and >enjoy using it most of the time. > >> "serious" is also a matter of opinion. I have some serious >> programmer friends who maintain, in complete sincerity, that >> serious programmers should not waste time on slow, script-kiddie >> languages like Python, but should be developing their skills >> with serious professional languages like Java, C#, etc. > >That is a narrow minded approach. different languages serve different purposes >and it's down to the developer to use which language she needs to achieve what >it is they've set out to do. Sometimes, basic shell scripts can be extremely >powerful for certain tasks; other needs will require something different. I >certainly wouldn't describe Python as a "script-kiddie" language. It's >extremely powerful and modern. So there ;-P lol Real programmers (can) write in assembler. -- http://mail.python.org/mailman/listinfo/python-list
Re: datetime module and timezone
in 671891 20120210 212545 Olive wrote: >In the datetime module, it has support for a notion of timezone but is >it possible to use one of the available timezone (I am on Linux). Linux >has a notion of timezone (in my distribution, they are stored >in /usr/share/zoneinfo). I would like to be able 1) to know the current >timezone and 2) to be able to use the timezone available on the system. >How can I do that? For 1) just type "date" on the command line. -- http://mail.python.org/mailman/listinfo/python-list
Re: Error with co_filename when loading modules from zip file
On Mon, Mar 05, 2012 at 02:22:55PM -0800, Vinay Sajip wrote: > On Mar 5, 8:36 pm, Bob wrote: > > > The logging package gets the filename and line number > > of the calling function by looking at two variables, the filename > > of the frame in the stack trace and the variable logging._srcfile. > > The comparison is done in logging/__init__.py:findCaller. > > > > The _srcfile is computed in logging/__init__.py - can you see which of > the paths it takes when computing _srcfile? Yes, it's this one, elif __file__[-4:].lower() in ['.pyc', '.pyo']: _srcfile = __file__[:-4] + '.py' _srcfile I beleive is correct. It's filename that isn't IMHO. > > I've tried putting only the pyc files, only the py files > > and both in the zip file. > > I think the filename info might be stored in the .pyc from when you > ran it outside the .zip. If you delete all .pyc files and only > have .py in the .zip, what happens? Nice one. I tried putting only pyc file, only py files and both in the zip. But I never tried putting the py files in the zip and deleting the pyc files. That makes it work as I'm guessing it has to recompile the python bytecode, making the filename and _srcfile match. The problem with this approach, is that it's less efficient to store the pyc files, since I've got to recompile them on startup, right? I found this issue, http://bugs.python.org/issue6811 and this related patch, http://hg.python.org/cpython/rev/5deb2094f033 that I think might address this issue. Although that's using 3.3 which isn't released yet. This is probably an issue that could be addressed in the logging library. Comparing the compiled in filename (which is determined at compile time) and the source file name (which is determined at module load time) doesn't seem to play well when you are moving the interpreter around in a zip file. I don't think one would expect it to. One solution would be to look to see if the filename ends in pythonNM[.zip]/logging/__init__.py Any suggestions? Thanks, Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Error with co_filename when loading modules from zip file
On Mon, Mar 05, 2012 at 02:22:55PM -0800, Vinay Sajip wrote: > On Mar 5, 8:36 pm, Bob wrote: > > > The logging package gets the filename and line number > > of the calling function by looking at two variables, the filename > > of the frame in the stack trace and the variable logging._srcfile. > > The comparison is done in logging/__init__.py:findCaller. > > > > The _srcfile is computed in logging/__init__.py - can you see which of > the paths it takes when computing _srcfile? > > > I've tried putting only the pyc files, only the py files > > and both in the zip file. > > I think the filename info might be stored in the .pyc from when you > ran it outside the .zip. If you delete all .pyc files and only > have .py in the .zip, what happens? Darn it, this was reported in 2007 http://bugs.python.org/issue1180193 and it was mentioned the logging package was effected. Yikes. Any resolutions? Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Error with co_filename when loading modules from zip file
On Tue, Mar 06, 2012 at 02:38:50AM -0800, Vinay Sajip wrote: > On Mar 6, 2:40 am, Bob Rossi wrote: > > > Darn it, this was reported in 2007 > > http://bugs.python.org/issue1180193 > > and it was mentioned the logging package was effected. > > > > Yikes. > > > > I will think about this, but don't expect any quick resolution :-( I > think the right fix would be not in the logging package, but in the > module loading machinery (as mentioned on that issue). > > I wouldn't worry about the performance aspect - once the logging > package is loaded, there's no performance impact. That's a tiny one- > off hit which you will probably not notice at all. OK. Do you know where the bytecode gets stored when you load a py file from a zip? My program can potentially run for hours, from an embedded context, and could call into the logger and other py files over and over. Are the bytecode files stored in RAM one time, or recomputed each time they are needed? Thanks, Bob -- http://mail.python.org/mailman/listinfo/python-list
Problem with time.time() standing still
Hi all, I've been a long time user of Python and written many extensions but this problem has me stumped. I've written a straight forward extension that wraps a vendors SDK for a video capture card. All works well except that in the Python thread on which I call the extension, after certain calls that I believe are using DirectShow, time stands still. The thread returns fine as the extension does its stuff in its own threads. In all other respects the Python thread seems unaffected but calls to time.time() always return the same time which is usually several seconds in the past or future and always has no fractional part. If I leave it long enough time will suddently jump forward after a few minutes, then again after a few minutes more. I've never encountered this behaviour before and can't understand what on earth is going on. If I call the 'C' time() function just the other side of my call to the extension the time is ticking along fine. It's just the one Python thread that is affected. If I call from the main thread then the main thread is affected. The calling program I've used to test this is just a few lines long. I believe the time function is a thin wrapper over the 'C' runtime so its very odd that time only stands still on the Python side but not on the 'C' side. As time is built in I've not looked at the code as its not in the distribution. Don't know if it would help to do that. This also can affect the time.sleep() function making it return immediately but that only seems to happen in my full application. Any ideas would be very greatly received. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with time.time() standing still
Thanks Daniel, that's interesting. Unfortunately there is no sensible code I can post because this only happens when I make a specific call into the vendors SDK. I can exercise my own code in the extension without a problem. The python test calling code is doing practically nothing. I make 3 calls to the extension to set things going and then loop every 5 seconds and print the time. I know Pythons own threading model is cooperative because of the GIL and therefore one thread can hog the show but I've never had issues with threads running in a C extension messing up a Python thread. I really need to understand what mechanism is at play here rather than work around it. Bob The time.clock() function does increment correctly. CPU is around 30% On 05/05/2012 21:17, Danyel Lawson wrote: > Add a time.sleep(0) call to all your loops. Multithreading in Python > is a cooperative cross platform threading simulation if you have tight > loops Python won't task switch until you make a system call. > Potentially preventing internal library variables from being updated. > > Your five minute interval may be almost exactly how long it takes to > process a flooded queue in a tight loop in your program and it may be > why it continues to happen as the queue waits to fill again while > processing happens. > > You can simulate the progression of time by overriding the time.time > function by simply setting it to a function that just increments a > module level or function property variable. You can also override the > time.time function to return the posix time function's value to maybe > get around whatever optimization is happening in the time.time > function to pass back the same value. > > If you post your sample code that exhibits the same behavior it may be > obvious to someone on the list as to what is the problem. > > > > On Sat, May 5, 2012 at 3:33 PM, Bob Cowdery wrote: >> Hi all, >> >> I've been a long time user of Python and written many extensions but >> this problem has me stumped. >> >> I've written a straight forward extension that wraps a vendors SDK for a >> video capture card. All works well except that in the Python thread on >> which I call the extension, after certain calls that I believe are using >> DirectShow, time stands still. The thread returns fine as the extension >> does its stuff in its own threads. In all other respects the Python >> thread seems unaffected but calls to time.time() always return the same >> time which is usually several seconds in the past or future and always >> has no fractional part. If I leave it long enough time will suddently >> jump forward after a few minutes, then again after a few minutes more. >> >> I've never encountered this behaviour before and can't understand what >> on earth is going on. If I call the 'C' time() function just the other >> side of my call to the extension the time is ticking along fine. It's >> just the one Python thread that is affected. If I call from the main >> thread then the main thread is affected. The calling program I've used >> to test this is just a few lines long. >> >> I believe the time function is a thin wrapper over the 'C' runtime so >> its very odd that time only stands still on the Python side but not on >> the 'C' side. As time is built in I've not looked at the code as its not >> in the distribution. Don't know if it would help to do that. >> >> This also can affect the time.sleep() function making it return >> immediately but that only seems to happen in my full application. >> >> Any ideas would be very greatly received. >> >> Bob >> -- >> http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with time.time() standing still
On 05/05/2012 23:05, Cameron Simpson wrote: > On 05May2012 20:33, Bob Cowdery wrote: > | I've written a straight forward extension that wraps a vendors SDK for a > | video capture card. All works well except that in the Python thread on > | which I call the extension, after certain calls that I believe are using > | DirectShow, time stands still. The thread returns fine as the extension > | does its stuff in its own threads. In all other respects the Python > | thread seems unaffected but calls to time.time() always return the same > | time which is usually several seconds in the past or future and always > | has no fractional part. > > Thought #1: you are calling time.time() and haven't unfortunately > renamed it? (I doubt this scenario, though the lack of fractional part > is interesting.) Not sure what you mean by renamed it. I also tried datetime and that had the same behaviour. > > | If I leave it long enough time will suddently > | jump forward after a few minutes, then again after a few minutes more. > | > | I've never encountered this behaviour before and can't understand what > | on earth is going on. If I call the 'C' time() function just the other > | side of my call to the extension the time is ticking along fine. It's > | just the one Python thread that is affected. > [...] > > Thought #2: On a UNIX system I'd be running the process under strace (or > dtrace or ktrace depending on flavour) to see what actual OS system calls are > being made during this behaviour. Is this feasible for you? I'm on Windows for this particular part of the application. The video SDK is Windows only. But debugging into this is probably the only way. I looked at the built-in time module. As far as I can tell it just calls the underlying C time() function as an extension. This is more mystifying as that is pretty much what I'm doing by checking the time in my own extension module where it ticks along fine. I have another scenario using a different part of their SDK where the time does still tick but the fractional part is frozen, not 0 but frozen. I'd dearly love to know what's going on here. I thought about extracting the time module, building it, calling it something else and seeing if it behaves the same. All time consuming unfortunately when I have a deadline. > > Cheers, -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with time.time() standing still
On 06/05/2012 00:11, Chris Angelico wrote: > On Sun, May 6, 2012 at 6:51 AM, Bob Cowdery wrote: >> The time.clock() function does increment correctly. CPU is around 30% > 30% of how many cores? If that's a quad-core processor, that could > indicate one core completely pegged plus a little usage elsewhere. It is a quad core but no CPU is pegged as there are a number of threads running in C. It's reasonably well spread. > > ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with time.time() standing still
On 06/05/2012 09:24, Chris Angelico wrote: > On Sun, May 6, 2012 at 6:18 PM, Bob Cowdery wrote: >> On 05/05/2012 23:05, Cameron Simpson wrote: >>> Thought #1: you are calling time.time() and haven't unfortunately >>> renamed it? (I doubt this scenario, though the lack of fractional part >>> is interesting.) >> Not sure what you mean by renamed it. I also tried datetime and that had >> the same behaviour. > In Python, names are nothing special, so you could do something like: > > time.time = lambda: 142857 > > which means that time.time() will forever return that constant. > > Take a snapshot of time.time early in your code somewhere, and try > using that instead, just in case. It's a long shot but might save you > some insanity! Thanks. I will try that. Away now until Monday when the battle will resume. Bob > ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with time.time() standing still
On 06/05/2012 09:49, Cameron Simpson wrote: > On 06May2012 09:18, Bob Cowdery wrote: > | On 05/05/2012 23:05, Cameron Simpson wrote: > | > On 05May2012 20:33, Bob Cowdery wrote: > | > | [...] calls to time.time() always return the same > | > | time which is usually several seconds in the past or future and always > | > | has no fractional part. > | > > | > Thought #1: you are calling time.time() and haven't unfortunately > | > renamed it? (I doubt this scenario, though the lack of fractional part > | > is interesting.) > | > | Not sure what you mean by renamed it. > > Like this: > > from time import time > [...] > time = some_unfortunate_other_function > [...] > now = time() # calls wrong function > > It seems unlikely, but possible. > > | I also tried datetime and that had > | the same behaviour. > > Makes my suggestion even less likely unless the time module itself gets > monkeypatched (i.e. "time.time = bad_function" somewhere). I don't think the function is subverted unless there is some way inside the vendor SDK or even DirectShow (the SDK uses this as a COM object) which can somehow hijack it. It does catch up every few minutes so there has to be a clue there. > | > | If I leave it long enough time will suddently > | > | jump forward after a few minutes, then again after a few minutes more. > | > | > | > | I've never encountered this behaviour before and can't understand what > | > | on earth is going on. If I call the 'C' time() function just the other > | > | side of my call to the extension the time is ticking along fine. It's > | > | just the one Python thread that is affected. > | > [...] > | > > | > Thought #2: On a UNIX system I'd be running the process under strace (or > | > dtrace or ktrace depending on flavour) to see what actual OS system calls > are > | > being made during this behaviour. Is this feasible for you? > | > | I'm on Windows for this particular part of the application. > > I'd guessed so. I've no Windows programming background; I was wondering > if there was something like this you had to hand on windows. Maybe, but not that I know of. > | [...] But debugging into this is probably the only way. I > | looked at the built-in time module. As far as I can tell it just calls > | the underlying C time() function as an extension. This is more > | mystifying as that is pretty much what I'm doing by checking the time in > | my own extension module where it ticks along fine. > > Hmm. A C extension I presume? How about writing a little pure Python > thread to call time.time(), and spin it off parallel to the rest of the > app; does it also get bad time.time() behaviour? I've tried that. It's only the Python thread (or Python main thread) that calls the extension that behaves like this. Other Python threads tick along fine with the correct time including a fractional part. > | I have another > | scenario using a different part of their SDK where the time does still > | tick but the fractional part is frozen, not 0 but frozen. > > Curiouser and curiouser. Indeed. Away now but battle will resume on Tuesday. Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Problem with time.time() standing still
Can anyone make sense of this.
I've looked over the Python timemodule.c again and it uses one of
gettimeofday(), ftime() or time(). The gettimeofday() is not available
on Windows so its going to use ftime() or time(). As time() only has a
resolution of 1 second and returns a long and I know Python time.time()
has a ms resolution it must be using ftime().
After going round the houses - a lot. I've made a trivially simple
extension that goes wrong. I can only think I must be doing something
incredibly stupid. Perhaps someone can put me out of my misery.
In the full app what happens is that it reports the correct time as
given by time() by gets the ms from somewhere also. When it goes wrong
it reports the same time as ftime().
Bob
>>> import mytime
>>> mytime.doTime()
TIME : 1336474857
FTIME secs 1336474880.00
FTIME ms 0.601000
FTIME secs+ms 1336474880.601000
0
>>> mytime.doTime()
TIME : 1336474871
FTIME secs 1336474880.00
FTIME ms 0.548000
FTIME secs+ms 1336474880.548000
0
>>> mytime.doTime()
TIME : 1336474897
FTIME secs 1336474880.00
FTIME ms 0.007000
FTIME secs+ms 1336474880.007000
0
The extension just prints out the time as got from time() and ftime().
For a start the two do not agree on what the time is. Secondly ftime()
stops incrementing while the python session is running. If I start a new
session it will have incremented.
>>> import mytime
>>> mytime.doTime()
TIME : 1336475029
FTIME secs 1336475008.00
FTIME ms 0.265000
FTIME secs+ms 1336475008.265000
0
Code and build file
#include "Python.h"
#include
#include
#include
#include
static struct timeb t;
static float secs = 0.0;
static float ms = 0.0;
static float both = 0.0;
static PyObject *doTime(
PyObject *self,
PyObject *args) {
time_t seconds;
seconds = time (NULL);
printf ("TIME : %ld\n", seconds);
ftime(&t);
secs = (float)t.time;
ms = (float)((float)t.millitm * 0.001);
printf("FTIME secs+ms %f\n", secs + ms);
return PyInt_FromLong((long)0);
}
static PyMethodDef pyInst_Methods[] = {
{"doTime",
doTime,
METH_VARARGS},
{NULL, NULL, 0, NULL}
};
#ifndef PyMODINIT_FUNC
#define PyMODINIT_FUNC void
#endif /* PyMODINIT_FUNC */
PyMODINIT_FUNC initmytime(void)
{
PyObject *module;
module = Py_InitModule3("mytime", pyInst_Methods,"Time module");
if (!module) return;
}
Build file
import os
from distutils.core import setup
from distutils.extension import Extension
if os.path.exists('C:\\Program Files (x86)'):
# 64 bit
ProgramFiles = 'Program Files (x86)'
else:
# 32 bit
ProgramFiles = 'Program Files'
setup(
name='Time Test',
author='Bob Cowdery',
ext_modules = [
Extension('mytime',
['pytime.cpp'],
include_dirs = ['C:\\' + ProgramFiles +
'\\Microsoft SDKs\\Windows\\v7.0A\\Include', 'C:\\Python26\\include'],
#define_macros = [("_AFXDLL", None)],
library_dirs = ['C:\\' + ProgramFiles +
'\\Microsoft SDKs\\Windows\\v7.0A\\Lib', 'C:\\Python26\\libs'],
libraries = ['User32', 'Gdi32', 'python26',
'odbc32', 'odbccp32']
)
]
)
On 06/05/2012 10:19, Bob Cowdery wrote:
On 06/05/2012 09:49, Cameron Simpson wrote:
On 06May2012 09:18, Bob Cowdery wrote:
| On 05/05/2012 23:05, Cameron Simpson wrote:
|> On 05May2012 20:33, Bob Cowdery wrote:
|> | [...] calls to time.time() always return the same
|> | time which is usually several seconds in the past or future and always
|> | has no fractional part.
|>
|> Thought #1: you are calling time.time() and haven't unfortunately
|> renamed it? (I doubt this scenario, though the lack of fractional part
|> is interesting.)
|
| Not sure what you mean by renamed it.
Like this:
from time import time
[...]
time = some_unfortunate_other_function
[...]
now = time() # calls wrong function
It seems unlikely, but possible.
| I also tried datetime and that had
| the same behaviour.
Makes my suggestion even less likely unless the time module itself gets
monkeypatched (i.e. "time.time = bad_function" somewhere).
I don't think the function is subverted unless there is some way inside
the vendor SDK or even DirectShow (the SDK uses this as a COM object)
which can somehow hijack it. It does catch up every few minutes so there
has to be a clue there.
|> | If I leave it long enough time will suddently
|> | jump forward after a few minutes, then again after a few minutes more.
|> |
|> | I&#x
Re: Problem with time.time() standing still
Hopefully somebody can add the last piece of this puzzle. My code didn't
work because I did make a silly mistake. The number of seconds since
EPOC is a large number but it also needs a high precision. Attempting to
put this value into a 32 bit float corrupts the least significant part
because 24 bits cannot hold that precision. Now Python floats are C
doubles and the calculation in timemodule.c is in doubles right back to
Python. Normally this retains the precision. For some inexplicable
reason when I make certain calls into this vendors SDK, ftime() is
getting precision problems and appears to be frozen as a result.
C, only supporting early binding cannot change the function referenced
at runtime so how the devil is it managing to do this.
On 08/05/2012 12:17, Bob Cowdery wrote:
Can anyone make sense of this.
I've looked over the Python timemodule.c again and it uses one of
gettimeofday(), ftime() or time(). The gettimeofday() is not available
on Windows so its going to use ftime() or time(). As time() only has a
resolution of 1 second and returns a long and I know Python
time.time() has a ms resolution it must be using ftime().
After going round the houses - a lot. I've made a trivially simple
extension that goes wrong. I can only think I must be doing something
incredibly stupid. Perhaps someone can put me out of my misery.
In the full app what happens is that it reports the correct time as
given by time() by gets the ms from somewhere also. When it goes wrong
it reports the same time as ftime().
Bob
>>> import mytime
>>> mytime.doTime()
TIME : 1336474857
FTIME secs 1336474880.00
FTIME ms 0.601000
FTIME secs+ms 1336474880.601000
0
>>> mytime.doTime()
TIME : 1336474871
FTIME secs 1336474880.00
FTIME ms 0.548000
FTIME secs+ms 1336474880.548000
0
>>> mytime.doTime()
TIME : 1336474897
FTIME secs 1336474880.00
FTIME ms 0.007000
FTIME secs+ms 1336474880.007000
0
The extension just prints out the time as got from time() and ftime().
For a start the two do not agree on what the time is. Secondly ftime()
stops incrementing while the python session is running. If I start a
new session it will have incremented.
>>> import mytime
>>> mytime.doTime()
TIME : 1336475029
FTIME secs 1336475008.00
FTIME ms 0.265000
FTIME secs+ms 1336475008.265000
0
Code and build file
#include "Python.h"
#include
#include
#include
#include
static struct timeb t;
static float secs = 0.0;
static float ms = 0.0;
static float both = 0.0;
static PyObject *doTime(
PyObject *self,
PyObject *args) {
time_t seconds;
seconds = time (NULL);
printf ("TIME : %ld\n", seconds);
ftime(&t);
secs = (float)t.time;
ms = (float)((float)t.millitm * 0.001);
printf("FTIME secs+ms %f\n", secs + ms);
return PyInt_FromLong((long)0);
}
static PyMethodDef pyInst_Methods[] = {
{"doTime",
doTime,
METH_VARARGS},
{NULL, NULL, 0, NULL}
};
#ifndef PyMODINIT_FUNC
#define PyMODINIT_FUNC void
#endif /* PyMODINIT_FUNC */
PyMODINIT_FUNC initmytime(void)
{
PyObject *module;
module = Py_InitModule3("mytime", pyInst_Methods,"Time module");
if (!module) return;
}
Build file
import os
from distutils.core import setup
from distutils.extension import Extension
if os.path.exists('C:\\Program Files (x86)'):
# 64 bit
ProgramFiles = 'Program Files (x86)'
else:
# 32 bit
ProgramFiles = 'Program Files'
setup(
name='Time Test',
author='Bob Cowdery',
ext_modules = [
Extension('mytime',
['pytime.cpp'],
include_dirs = ['C:\\' + ProgramFiles +
'\\Microsoft SDKs\\Windows\\v7.0A\\Include', 'C:\\Python26\\include'],
#define_macros = [("_AFXDLL", None)],
library_dirs = ['C:\\' + ProgramFiles +
'\\Microsoft SDKs\\Windows\\v7.0A\\Lib', 'C:\\Python26\\libs'],
libraries = ['User32', 'Gdi32', 'python26',
'odbc32', 'odbccp32']
)
]
)
On 06/05/2012 10:19, Bob Cowdery wrote:
On 06/05/2012 09:49, Cameron Simpson wrote:
On 06May2012 09:18, Bob Cowdery wrote:
| On 05/05/2012 23:05, Cameron Simpson wrote:
|> On 05May2012 20:33, Bob Cowdery wrote:
|> | [...] calls to time.time() always return the same
|> | time which is usually several seconds in the past or future
and always
|> | has no fractional part.
|>
|> Thought #1: you are calling time.time() and haven't unfortunately
|> renamed it? (I doubt this scenario, though the lack of
fractional part
|> is interest
Re: Problem with time.time() standing still
Hi Roel "Our problem turned out to be caused by a loss of precision in an application of ours, caused by Direct3D. The solution for us was to include the flag D3DCREATE_FPU_PRESERVE in CreateDevice(). The documentation seems to imply that the lower precision only has effect in the Direct3D code, but in reality it lowers precision in the rest of the code too (the whole process or the whole thread, I'm not sure). " That is spot on and very likely the cause. Thanks very much for responding. I do have some control over the vendor and have passed that information on to them. Hopefully they will investigate and fix the problem. Bob -- http://mail.python.org/mailman/listinfo/python-list
Hashability questions
Noob alert: writing my first Python class library. I have a straightforward class called Utility that lives in Utility.py. I'm trying to get a handle on best practices for fleshing out a library. As such, I've done the following for starters: def __str__(self): return str(type(self)) # def __eq__(self,other): #return hash(self) == hash(other) The commented-out method is what I'm questioning. As-is, I can do the following from my test harness: u = Utility() print(str(u)) print(hash(u)) u2 = Utility() print(hash(u2)) print(hash(u) == hash(u2)) However if I uncomment the above _eq_() implementation, I get the following output: Traceback (most recent call last): File "/Users/bob/PycharmProjects/BGC/Tests.py", line 7, in print(hash(u)) TypeError: unhashable type: 'Utility' Process finished with exit code 1 Obviously there is some sort of default implementation of __hash__() at work and my implementation of _eq_() has somehow broken it. Can anyone explain what's going on? -- http://mail.python.org/mailman/listinfo/python-list
Re: Hashability questions
On Monday, May 14, 2012 8:35:36 PM UTC-5, alex23 wrote: > It looks like this has changed between Python 2 and 3: > > "If a class does not define an __eq__() method it should not define a > __hash__() operation either; if it defines __eq__() but not > __hash__(), its instances will not be usable as items in hashable > collections." > > From: http://docs.python.org/dev/reference/datamodel.html#object.__hash__ > > You should just be able to add a __hash__ to Utility and it'll be fine. Thanks, Alex. I should have mentioned I was using Python 3. I guess all this is a bit over-thought to just crank out some code -- in practice, comparing two classes for equality is mostly YAGNI -- but it's my way of coming to a reasonably in-depth understanding of how things work ... -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Book for a C Programmer?
On Thu, May 24, 2012 at 8:28 AM, Javier Novoa C. wrote: > On 2012-05-24, alister wrote: >> On Wed, 23 May 2012 16:45:05 -0700, hsaziz wrote: >> >>> I am trying to join an online class that uses python. I need to brush up >>> on the language quickly. Is there a good book or resource that covers it >>> well but does not have to explain what an if..then..else statement is? >>> >>> Thanks. >> >> Dive into python seems to be quite popular & can be read online fro free >> >> > > Learning Python by Mark Lutz, from O'Reilly is a good one, I've been > reading it and it also enforces a comparison between C programming and > Python. However, it's focused on Python 2, but it also mentions Python > 3 things in the text... > There is a new edition out, copyright 2010 if I recall correctly, that updates its coverage to Python 3.x, which is the book's primary focus, though it points out where 3.x syntax does not work in version 2.x. Cheers! boB -- http://mail.python.org/mailman/listinfo/python-list
PyImport_ImportModule bug in 3.1.2
Hi:
My Python version is 3.1.2.
I am programming embedding c with python in windows.
When I imported urllib.request in my test py file,
PyImport_ImportModule always return NULL.
But I imported re or cmd ,they work fine.
I found urllib is folder, and request seems to be submodule.
I don't know how to import it. I confirmed myutil.py and application
are in same folder.
Could you give me some advice?
Thank you.
myutil.py
import sys
import urllib.request
CRLF = '\r\n'
def addValue(oper1,oper2):
return (oper1 + oper2);
def ConnectURL(url):
try:
req = urllib.request.urlopen(url)
response = req.read()
req.close()
return (req.getcode(),req.geturl())
except BaseException as exp:
sys.stdout.write(exp)
native code:
int APIENTRY _tWinMain(HINSTANCE hInstance,
HINSTANCE hPrevInstance,
LPTSTRlpCmdLine,
int nCmdShow)
{
PyObject* pPyUtil = NULL;
PyObject* pPypfn= NULL;
PyObject* pPyArgs = NULL;
PyObject* pPyRet= NULL;
int nRet= 0;
int nValue = 0;
Py_Initialize();
PyImport_ImportModule("sys"); // ok
PyImport_ImportModule("csv"); // ok
PyImport_ImportModule("urllib");// ok
pPyUtil = PyImport_ImportModule("myutil"); // failed ,
if(pPyUtil != NULL)
{
Py_DECREF(pPyUtil);
}
Py_Finalize();
return 0;
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: 'super' object has no attribute '__setitem__'
I'm pretty sure I'd actually read the first 2 links you point to, but the
difference between __setattr__ and __setitem__ still never registered with
me -- perhaps partly because even the discussion of __setattr__ discusses
adding an entry to the "*dictionary* of instance attributes".
*MANY* thanks for your help!
On Thu, Aug 18, 2011 at 10:07 PM, Eric Snow wrote:
> On Thu, Aug 18, 2011 at 7:44 PM, luvspython wrote:
> > I'm using Python 2.7 and the code below fails at the 'super' statement
> > in the __setitem__ function in the HistoryKeeper class. The error is:
> > 'super' object has no attribute '_setitem__'
> >
> > Can anyone please tell me why and how to fix it? (I've googled
> > endlessly and I don't see the problem.)
> >
> > [The code will seem silly as it is, because it's pared down to show
> > the example. The goal is that multiple classes, like the Vehicle
> > class below, will inherit HistoryKeeper. History keeper overloads
> > __setitem__ and will eventually keep a running history every time an
> > attribute of any of the inheriting classes is changed.]
> >
> > Thanks in advance
> >
> >
> > class HistoryKeeper(object):
> >def __init__(self, args):
> >for arg, value in args.items():
> >if arg != 'self':
> >self.__setitem__(arg, value)
> >
> >def __setitem__(self, item, value):
> >super(HistoryKeeper, self).__setitem__(item, value)
> >
> >
> > class Vehicle(HistoryKeeper):
> >def __init__(self, tag, make, model):
> >args = locals()
> >super(Vehicle, self).__init__(args)
> >
> >
> > if __name__ == "__main__":
> >car = Vehicle('TAG123', 'FORD', 'Model A')
> >print car.make
>
> Did you mean to use __setattr__ instead? object, the base class of
> HistoryKeeper, does not have a __setitem__ method, hence the
> AttributeError. super() is a proxy for the next class in the MRO,
> typically the base class of your class.
>
> Keep in mind that is equivalent to
> . However,
> is equivalent to .
>
> see:
>
> http://docs.python.org/reference/datamodel.html#object.__setattr__
> http://docs.python.org/reference/datamodel.html#object.__setitem__
> http://docs.python.org/library/functions.html#super
>
> -eric
>
> > --
> > http://mail.python.org/mailman/listinfo/python-list
> >
>
--
http://mail.python.org/mailman/listinfo/python-list
Setting the time in Win7
Permissions! We're running in an account as an administrator (the only account on the laptops) and the program just calls system(time ) and system(date ) after reading it from a connected GPS receiver. I've fiddled with everything I could find in the registry and with the secpol.msc program and there doesn't seem to be any reason that it can't set the time, but it can't. Any ideas? Thanks! Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Setting the time in Win7
Hi! Yup. I've been from one end of that article to the other with no luck. It must be something to do with the shell that the system() call creates not having permission to set the time, but I can't figure out how to get around it. Just using the GUI the account where the user is running the program from has permission to set the time. On 2011-08-22 11:41:45 -0600, Tim Golden said: If memory serves, you need to enable a specific privilege to set the time in Vista+. Just a moment... Have a look here: http://support.microsoft.com/kb/300022 and look for SeSystemtimePrivilege generally. Sorry; I'm a bit rushed at the moment. Feel free to post back if that isn't clear TJG On 22/08/2011 17:35, Bob Greschke wrote: Permissions! We're running in an account as an administrator (the only account on the laptops) and the program just calls system(time ) and system(date ) after reading it from a connected GPS receiver. I've fiddled with everything I could find in the registry and with the secpol.msc program and there doesn't seem to be any reason that it can't set the time, but it can't. Any ideas? Thanks! Bob -- http://mail.python.org/mailman/listinfo/python-list
Re: Setting the time in Win7
Several people have been hacking away on this computer we are testing
on, so I'm not sure what settings -- other than all of them -- have
been messed with, but popen("time ...") seems to work, but system("time
...") does not. I'm going to restore the machine to its original state
and see what happens.
Bob
On 2011-08-22 11:41:45 -0600, Tim Golden said:
If memory serves, you need to enable a specific privilege to
set the time in Vista+. Just a moment...
Have a look here:
http://support.microsoft.com/kb/300022
and look for SeSystemtimePrivilege generally. Sorry; I'm
a bit rushed at the moment. Feel free to post back if
that isn't clear
TJG
On 22/08/2011 17:35, Bob Greschke wrote:
Permissions!
We're running in an account as an administrator (the only account on the
laptops) and the program just calls system(time ) and system(date
) after reading it from a connected GPS receiver. I've fiddled with
everything I could find in the registry and with the secpol.msc program
and there doesn't seem to be any reason that it can't set the time, but
it can't. Any ideas?
Thanks!
Bob
--
http://mail.python.org/mailman/listinfo/python-list
Re: Setting the time in Win7
On 2011-08-23 02:26:38 -0600, Tim Golden said:
On 22/08/2011 20:42, Bob Greschke wrote:
Several people have been hacking away on this computer we are testing
on, so I'm not sure what settings -- other than all of them -- have been
messed with, but popen("time ...") seems to work, but system("time ...")
does not. I'm going to restore the machine to its original state and see
what happens.
Hoping that this helps: you can programatically set the system time
from within Python by using the pywin32 modules, or ctypes if you
prefer. The code below works for an already-elevated command prompt
by enabling the SystemTime privilege and (crudely) moving the time
forward by five minutes by way of showing what's happening before
resetting it back.
I've commented out the actual SetSystemTime calls just in case anyone
cuts-and-pastes indjudiciously. Ideally you should disable the
privilege afterwards but I've left that out so as not to clutter
the example.
import os, sys
import win32api
import win32security
import ntsecuritycon
hToken = win32security.OpenProcessToken (
win32api.GetCurrentProcess (),
ntsecuritycon.MAXIMUM_ALLOWED
)
time_privilege = win32security.LookupPrivilegeValue (None,
win32security.SE_SYSTEMTIME_NAME)
win32security.AdjustTokenPrivileges (
hToken, 0,
[(time_privilege, win32security.SE_PRIVILEGE_ENABLED)]
)
current_time = win32api.GetSystemTime ()
print "Current time:", current_time
new_time = list (current_time)
new_time[5] += 5
## print win32api.SetSystemTime (*new_time)
print "Current time:", win32api.GetSystemTime ()
## print win32api.SetSystemTime (*current_time)
print "Current time:", win32api.GetSystemTime ()
TJG
Oooo. Now I can be dangerous. We kinda also solved the whole thing
for us (just a few minutes ago) by checking the checkbutton "Run as
administrator" in the Properties, Compatibility tab, for python.exe and
pythonw.exe. The account is an Administrator, so it's OK for this.
I thought there must be a way through pywin32, but I don't know much of
anything about Windows API calls. I have a Windows Programming book
collecting dust somewhere...
Thanks!
Bob
--
http://mail.python.org/mailman/listinfo/python-list
my computer is allergic to pickles
I'm using python to do some log file analysis and I need to store on disk a very large dict with tuples of strings as keys and lists of strings and numbers as values. I started by using cPickle to save the instance of the class that contained this dict, but the pickling process started to write the file but ate so much memory that my computer (4 GB RAM) crashed so badly that I had to press the reset button. I've never seen out-of-memory errors do this before. Is this normal? (I know from the output that got written before the crash that my program had finished building the dict and started the pickle. When I tried running the other program that reads the pickle and analyzes the data in it, it gave an error because the file was incomplete. So I know where in my code the crash happened.) >From searching the web, I get the impression that pickle uses a lot of memory because it checked for recursion and other things that could break other serialization methods. So I've switched to using marshal to save the dict itself (the only persistent thing in the class, which just has convenience methods for adding data to the dict and searching it for the second stage of analysis). I found some references to h5 tables for getting around the pickling memory problem, but I got the impression they only work with fixed columns, not a somewhat complex data structure like mine. Any comments, suggestions? -- http://mail.python.org/mailman/listinfo/python-list
Re: my computer is allergic to pickles
Miki Tebeka wrote: > > I'm using python to do some log file analysis and I need to store > > on disk a very large dict with tuples of strings as keys and > > lists of strings and numbers as values. > I recommend that you'll use the shelve module. It stores data on disk and is > more memory efficient than in-memory pickle objects. OK, I got this to work with marshal. What makes shelve better? Or, which situations does shelve suit better and which does marshal suit better? -- http://mail.python.org/mailman/listinfo/python-list
Re: my computer is allergic to pickles
GSO wrote: > On 5 March 2011 02:14, MRAB wrote: > ... > >> Any comments, suggestions? > >> > > You obviously can't feed your computer pickles then. > > How about a tasty tidbit of XML? Served up in a main dish of DOM, or > serially if preferred? Well, right now it takes three lines to save the dict object: data_file = open(data_filename, 'wb') marshal.dump(analysis, file, 2) data_file.close() and three to load it. I doubt I could do it that easily in XML _and_ the data file would be enormous. (XML always is, let's be honest. The file doesn't need to be human readable or editable.) -- http://mail.python.org/mailman/listinfo/python-list
Re: my computer is allergic to pickles
MRAB wrote: > On 05/03/2011 01:56, Bob Fnord wrote: > > I'm using python to do some log file analysis and I need to store > > on disk a very large dict with tuples of strings as keys and > > lists of strings and numbers as values. > > > > I started by using cPickle to save the instance of the class that > > contained this dict, but the pickling process started to write > > the file but ate so much memory that my computer (4 GB RAM) > > crashed so badly that I had to press the reset button. I've never > > seen out-of-memory errors do this before. Is this normal? > > > > (I know from the output that got written before the crash that my > > program had finished building the dict and started the > > pickle. When I tried running the other program that reads the > > pickle and analyzes the data in it, it gave an error because the > > file was incomplete. So I know where in my code the crash > > happened.) > > > >> From searching the web, I get the impression that pickle uses a > > lot of memory because it checked for recursion and other things > > that could break other serialization methods. So I've switched to > > using marshal to save the dict itself (the only persistent thing > > in the class, which just has convenience methods for adding data > > to the dict and searching it for the second stage of analysis). > > > > I found some references to h5 tables for getting around the > > pickling memory problem, but I got the impression they only work > > with fixed columns, not a somewhat complex data structure like > > mine. > > > > Any comments, suggestions? > > > Would a database work? I want a portable data file (can be moved around the filesystem or copied to another machine and used), so I don't want to use mysql or postgres. I guess the "sqlite" approach would work, but I think it would be difficult to turn the tuples of strings and lists of strings and numbers into database table lines. Would a database in a file have any advantages over a file made by marshal or shelve? I'm more worried about the fact that a python program in user space can bring down the computer! -- http://mail.python.org/mailman/listinfo/python-list
Re: my computer is allergic to pickles
Miki Tebeka wrote:
> > Or, which situations does shelve suit better and which does
> > marshal suit better?
> shelve ease of use and the fact it uses the disk to store objects makes it a
> good choice if you have a lot of object, each with a unique string key (and a
> tuple of strings can be converted to and from a string).
>
> db = shelve.open("/tmp/foo.db")
> db["key1"] = (1, 2, 3)
> ...
>
> Marshal is faster and IIRC more geared toward network operations. But I
> haven't used it that much ...
>From looking at the shelve info in the library reference, I get
the impression it's tricky to change the values in the dict for
existing keys and be sure they get changed on disk. My dict lists
of strings and integers as values and the lists get changed as
the program analyzes the input files, then stored on disk in
their final form. I guess marshal is better for that.
How can you convert a tuple of strings to a string and back in a
reliable deterministic way? The original strings may have ' " ,
in them.
--
http://mail.python.org/mailman/listinfo/python-list
Re: my computer is allergic to pickles
"Martin P. Hellwig" wrote: > On 05/03/2011 01:56, Bob Fnord wrote: > > > Any comments, suggestions? > > > No but I have a bunch of pseudo-questions :-) > > What version of python are you using? How about your OS and bitspace > (32/64)? Have you also tried using the non-c pickle module? If the data > is very simple in structure, perhaps serializing to CSV might be an option? python 2.6.6 ubuntu 64 bit The library ref says cPickle is "optimized" and "up to 1000 times faster than pickle" but of course doesn't mention memory. The data to save (and load in another python program) is a dict with keys = tuples of strings (including " ' , and other troublesome characters) and values = lists of strings and integers. As the 1st program runs, it adds new keys AND changes the contents of the value lists. (The 2nd program only reads the dict into memory, analyzes it, and prints to STDOUT.) -- http://mail.python.org/mailman/listinfo/python-list
Re: my computer is allergic to pickles
Terry Reedy wrote: > On 3/7/2011 4:50 AM, Bob Fnord wrote: > > > I want a portable data file (can be moved around the filesystem > > or copied to another machine and used), > > Used only by Python or by other software? just Python > > Would a database in a file have any advantages over a file made > > by marshal or shelve? > > If you have read the initial paragraphs of the marshal doc and your > needs fit within its limitations, go ahead and use it. (Also note that > Python could switch to a new version in the future.) OK, I think marshal is just what I need. > Keyed databases have the advantage that you can change the data file. If > you do not need to do that (as opposed to read in, do whatever, and > write out in entirety) then that is no advantage to you. OK, thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: my computer is allergic to pickles
Peter Otten <__pete...@web.de> wrote: > Bob Fnord wrote: > > I started by using cPickle to save the instance of the class that > > contained this dict, but the pickling process started to write > > the file but ate so much memory that my computer (4 GB RAM) > > crashed so badly that I had to press the reset button. I've never > > seen out-of-memory errors do this before. Is this normal? snipped myself > > Any comments, suggestions? > > Have you seen that one? > > http://mail.python.org/pipermail/python-list/2008-July/1139855.html Not until now, but that's interesting. But I didn't even get a backtrace, just a totally locked up computer! -- http://mail.python.org/mailman/listinfo/python-list
Re: my computer is allergic to pickles
Miki Tebeka wrote: > > >From looking at the shelve info in the library reference, I get > > the impression it's tricky to change the values in the dict for > > existing keys and be sure they get changed on disk. > You can use writeback=True or call sync at the right places. > > > > How can you convert a tuple of strings to a string and back in a > > reliable deterministic way? The original strings may have ' " , > > in them. > You can use marshal, json or any other serializing library. Thanks for the tips. I guess I'll use json if I ever need to be able to read the file with something else besides python, but marshal seems fine for now. -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
in 654905 20110408 171055 Ethan Furman wrote: >Westley Mart�nez wrote: >> On Fri, 2011-04-08 at 01:41 -0500, harrismh777 wrote: >>> >>> Freedom isn't free... you have to fight for it... always. >> >> Why should a business listen to you? You're not gonna buy any software >> anyways. >> > >From a thread a few months back I can say there are a couple companies >with posters on this list that are successful in supporting *and >selling* open-source software. IBM fits that description ... -- http://mail.python.org/mailman/listinfo/python-list
can't get urllib2 or httplib to work with tor & privoxy
Here's my python code:
import httplib, urllib2
proxy_handler = {'http' : 'localhost:8118',
'https' : 'localhost:8118'}
def connect_u2(url = 'http://ipid.shat.net/iponly/'):,
proxied = urllib2.ProxyHandler(proxy_handler)
opnr = urllib2.build_opener(proxied)
opnr.addheaders = [('User-agent', agent)]
rsp = opnr.open(url)
page = rsp.read()
return page
def connect_h(url = 'http://ipid.shat.net/iponly/'):
cnn = httplib.HTTPConnection('127.0.0.1', 8118)
cnn.connect()
cnn.request('GET', url)
rsp = cnn.getresponse()
stderr.write('%u %s\n' % (rsp.status, rsp.reason))
page = rsp.read(500)
cnn.close()
return page
Both methods give me a 503 error, even though this
export http_proxy='http://localhost:8118/'
lynx -dump 'http://ipid.shat.net/iponly/'
works in bash and prints out the IP of a tor exit.
What am I doing wrong in python? (I tried both 'localhost:8118' and
'http://localhost:8118/' in the proxy_handler dict.)
--
http://mail.python.org/mailman/listinfo/python-list
Re: can't get urllib2 or httplib to work with tor & privoxy
Chris Angelico wrote: > On Tue, May 10, 2011 at 4:20 AM, Bob Fnord wrote: > > Both methods give me a 503 error... > > As a networking geek, my first thought would be to fire up a tiny > little "snoop server" and see what, exactly, the two methods are > doing. (Ignore the HTTPS options as they're more complicated to snoop > on; you're almost certainly going to see exactly the same on the > HTTP.) > > A snoop server is simply an HTTP server that dumps its requests to > stdout or a file, utterly raw. You can write one in Python fairly > easily, or use another program; if I'm working on Windows, I'll > generally use my own RosMud MUD client, but you'll be able to whip up > a cross-platform one in half a page of code in any decent high level > language. I recommend you work at the sockets level, rather than using > an HTTP library, unless you have one that can emit the entire request, > exactly as it came from the client. > > Once you've found the difference(s) between Lynx and your script, you > can see what's causing the 503 (Service Unavailable) error; it may be > that you need to authenticate with the proxy. Duh, I was doing something else stupid. Thanks but never mind! -- http://mail.python.org/mailman/listinfo/python-list
Re: detaching comprehensions
I don't know whether you wanted a reply, since you did not ask for one. I am not even sure what your point is. See other comments below. On 9/8/2017 4:24 PM, Stefan Ram wrote: Maybe you all know this, but to me this is something new. I learnt it by trial and error in the Python 3.6.0 console. Most will know list comprehensions: |>>> [ i for i in range( 3, 5 )] |[3, 4] I found out that the comprehension can be detached from the list: |>>> k =( i for i in range( 3, 5 )) but one must use an extra pair of parentheses around it in the assignment. Now I can insert the "generator" »k« into a function call, but a spread operator should cannot be used there. |>>> sum( k ) |7 »sum« expects exactly two arguments, and this is what »k« provides. Where did you get that idea. If you look at the docs you will see: "sum(iterable[, start]) Sums start and the items of an iterable from left to right and returns the total. start defaults to 0." sum expects 1 or 2 arguments; when you write sum(k) you are providing 1 argument. But to insert it again into the place where it was "taken from", a spread operator is required! |>>> k =( i for i in range( 3, 5 )) |>>> [ *k ] |[3, 4] "taken from"?? k is a generator object. Clear? Bob Gailer -- https://mail.python.org/mailman/listinfo/python-list
Re: Questions.
On Fri, Sep 8, 2017 at 9:54 PM, Michael Torrie wrote: > On 09/08/2017 08:35 PM, V Vishwanathan wrote: >> Hi, From what I see in the recent 4/5 digests, this forum seems to be for >> advanced >> >> and professional programmers. >> >> So wondering if a newbie can post some questions to understand errors in his >> code >> >> or will it look silly? > > Yes you may indeed post here. There's also a beginners list called > python-help, which may seem less intimidating. That would be Python Tutor. Subscription information may be found at https://mail.python.org/mailman/listinfo/tutor It is a moderated list, so there may be a short delay before your first post(s) may come through. -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: "comprehend" into a single value
On 10/7/2017 11:17 PM, Nathan Hilterbrand wrote:
dict= {10: ['a',1,'c'], 20: ['d',2,'f']}
p = sum([dict[i][1] for i in dict])
Something like that?
Ah, but that's 2 lines.
sum(val[1] for val in {10: ['a',1,'c'], 20: ['d',2,'f']}.values())
On Sat, Oct 7, 2017 at 11:07 PM, Andrew Z wrote:
Hello,
i wonder how can i accomplish the following as a one liner:
dict= {10: ['a',1,'c'], 20: ['d',2,'f']}
p = 0
for i in dict:
p += dict[i][1]
Thank you
--
https://mail.python.org/mailman/listinfo/python-list
--
Image and video hosting by TinyPic
--
https://mail.python.org/mailman/listinfo/python-list
Re: The "loop and a half"
On Sun, Oct 8, 2017 at 5:36 AM, bartc wrote: > > On 08/10/2017 10:12, Steve D'Aprano wrote: >> >> On Sun, 8 Oct 2017 02:06 am, bartc wrote: >>> I'm getting fed up with this thread now. >> >> >> This thread would be a lot less frustrating if you would enter into it with a >> spirit of open-minded enquiry rather than an overbearing sense of superiority >> that anything others do that you don't must be worthless. > > > Frustrating for whom? > > It seems to me that it's pretty much everyone here who has an overbearing > sense of superiority in that everything that Unix or Linux does is a million > times better than anything else. I follow this list in an effort to learn as much as I can even though I am usually a fish out of water here. But this thread in all its twists and turns and various subject line changes seems to have gotten totally out of hand. Even though I am quoting only part of this one message, there are actually many others that I am responding to here. In my opinion (Honestly admitting my lack of technical competence.), this insatiable thirst on this list to get every bit of technical minutiae exactly correct is doing a severe disservice to the friendliness of this community. Yes, gently correct the (perceived) errors, because we all want to have it right, but please cease this incessant pounding of points (and people) into the ground to prove we are right and they are wrong! I doubt any of this is going to change Bart's mind. Why can we not allow him to make his points, respond to them appropriately, and then let it go when it is clear he has strongly held opinions that are not likely to change? And Bart, when large numbers of technical experts in their fields have spent many hours/months/years, yea, even several decades, developing these software systems, why do you think that you, all by yourself, know better? Can you not see how frustrating this is for people who have spent good chunks of their lives trying to do the best they can on these software systems? Don't you think it is a better approach to perhaps do some self-examination and approach things from a more humble learner's perspective? And BTW, there are many users of non-*nix systems on this list, or who do work on multiple operating system platforms. Can we not let people be who they are, perceived warts (valid or not) and all, and after responding (hopefully gently) to technical errors just let them be??? Peace. -- boB -- https://mail.python.org/mailman/listinfo/python-list
OT: MPC-HC project ending? [Re: Lies in education [was Re: The "loop and a half"]]
On Wed, Oct 11, 2017 at 4:45 PM, Mikhail V wrote: > > PS Off-topic: > I have a related observation regarding popularity of software. > There is such a program "VLC", which is a video player. Some would > think it is sort of best free player, etc. I was also under impression, > but then I've found a video player called "MPC-HC". > I have started to use it and quickly understood that it is by far more > superior player than VLC, literally by all core parameters - performance, > codec support, customizable key bindings with a lot of internal commands. > (Remark for Linux users: there is no Linux version of that player, it > is Windows-only) I went to https://mpc-hc.org/2017/07/16/1.7.13-released-and-farewell/ and it says: July 16, 2017 XhmikosR v1.7.13, the latest, and probably the last release of our project… For quite a few months now, or even years, the number of active developers has been decreasing and has inevitably reached zero. This, unfortunately, means that the project is officially dead and this release would be the last one. …Unless some people step up that is. So, if someone’s willing to really contribute and has C/C++ experience, let me know on IRC or via e-mail. Otherwise, all things come to an end and life goes on. It’s been a nice journey and I’m personally pretty overwhelmed having to write this post. Thanks to everyone who has contributed in any way all these years; Remember, MPC-HC is an 11-year old project. -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 2 -> 3, urllib.urlOpen
On Fri, Oct 13, 2017 at 5:27 PM, Irv Kalb wrote: > I've looked at the Python 3.6 documentation for urllib, and I see that > certain calls have been changed and others have been eliminated. But my eyes > glaze over trying to figure out what to use instead. > > My question is: Is there a simple (hopefully one or two line) replacement for > my call to url lib.urlopen().read() I don't know much about this, but might you be looking for urllib.request.urlopen()? See https://docs.python.org/3/library/urllib.request.html#module-urllib.request in the docs. > I know that there are other modules out there that handle requests (like the > Requests module), but this is a strictly controlled university environment. > I cannot download any external packages (other then pygame, which I got > special permission for) onto the computers in the school. Therefore, I'm > looking for something in the Python 3.6 Standard Library. The above is in the standard library. -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Python noob having a little trouble with strings
On Thu, Oct 26, 2017 at 9:25 PM, wrote: > If s1 = "Welcome students", what is the output when you print the following: > > 1. s4 = 3 * s1 > > 2. s1[3 : 6] > > 3. 'W' in s1 > > 4. S1[-1] > > 5. S1[:-1] > > Any help would be great, thanks! Why not find out for yourself and print these in the Python interpreter? For instance: > py Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. py3: s1 = "Welcome students" py3: s4 = 3 * s1 py3: print(s4) Welcome studentsWelcome studentsWelcome students -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: General Purpose Pipeline library?
On Nov 20, 2017 10:50 AM, "Jason" wrote:
>
> a pipeline can be described as a sequence of functions that are applied
to an input with each subsequent function getting the output of the
preceding function:
>
> out = f6(f5(f4(f3(f2(f1(in))
>
> However this isn't very readable and does not support conditionals.
>
> Tensorflow has tensor-focused pipepines:
> fc1 = layers.fully_connected(x, 256, activation_fn=tf.nn.relu,
scope='fc1')
> fc2 = layers.fully_connected(fc1, 256, activation_fn=tf.nn.relu,
scope='fc2')
> out = layers.fully_connected(fc2, 10, activation_fn=None, scope='out')
>
> I have some code which allows me to mimic this, but with an implied
parameter.
>
> def executePipeline(steps, collection_funcs = [map, filter, reduce]):
> results = None
> for step in steps:
> func = step[0]
> params = step[1]
> if func in collection_funcs:
> print func, params[0]
> results = func(functools.partial(params[0],
*params[1:]), results)
> else:
> print func
> if results is None:
> results = func(*params)
> else:
> results = func(*(params+(results,)))
> return results
>
> executePipeline( [
> (read_rows, (in_file,)),
> (map, (lower_row, field)),
> (stash_rows, ('stashed_file', )),
> (map, (lemmatize_row, field)),
> (vectorize_rows, (field, min_count,)),
> (evaluate_rows, (weights, None)),
> (recombine_rows, ('stashed_file', )),
> (write_rows, (out_file,))
> ]
> )
>
> Which gets me close, but I can't control where rows gets passed in. In
the above code, it is always the last parameter.
>
> I feel like I'm reinventing a wheel here. I was wondering if there's
already something that exists?
IBM has had for a very long time a program called Pipelines which runs on
IBM mainframes. It does what you want.
A number of attempts have been made to create cross-platform versions of
this marvelous program.
A long time ago I started but never completed an open source python
version. If you are interested in taking a look at this let me know.
--
https://mail.python.org/mailman/listinfo/python-list
adding elif to for
Has any thought been given to adding elif to the for statement? for x in foo: if y: break elif a==b: something else: something else as a shortcut to: for x in foo: if y: break else: if a==b: something else: something else bob gailer -- https://mail.python.org/mailman/listinfo/python-list
Re: Linux/Windows GUI programming: GUI-fy a CLI using pyInstaller
in 788357 20180105 132921 Kevin Walzer wrote: >On 1/1/18 11:45 AM, X. wrote: >> Ulli Horlacher: >>> I have to transfer a python 2.7 CLI programm into one with a (simple) GUI. >>> The program must run on Linux and Windows and must be compilable with >>> pyinstall, because I have to ship a standalone windows.exe >>> Any kind of installer is not acceptable. >>> >>> Reading https://github.com/pyinstaller/pyinstaller/wiki/Supported-Packages >>> supported GUI packages are PyGTK, PyQt4, PyQt5, wxPython >>> I have tested tkinter by myself and it works, too. >>> I do not like GTK and Qt, because they are too complex. >>> >>> I want to do VERY simple things and I prefer a simple GUI toolkit :-) >> >> >> me too ! >> > >Try easygui: > >https://pypi.python.org/pypi/easygui Looks like Zenity. -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Syntax error
On 2/3/2018 2:40 PM, Terry Reedy wrote: On 2/3/2018 2:10 PM, Kevin Doney wrote: Hi. *pip3 install --upgrade tensorflow-gpu* When I try the above command I get SyntaxError: invalid syntax Please help. This group helps those who help the group -- by asking questions with sufficient information to answer. Post the ENTIRE trackback. It also helps us to know: operating system (e.g., windows 10) what does "when I try" mean? (e.g. at a windows command prompt I entered ...) What the *'s are for best is to copy the entire session and paste it into your email. Example Microsoft Windows [Version 10.0.16299.192] (c) 2017 Microsoft Corporation. All rights reserved. C:\Users\bgailer>pip install foo Collecting foo Could not find a version that satisfies the requirement foo (from versions: ) No matching distribution found for foo We could presume in your case that you tried to enter your pip command in a python interactive session. For example: C:\Users\bgailer>python Python 3.3.5 (v3.3.5:62cf4e77f785, Mar 9 2014, 10:35:05) [MSC v.1600 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> pip install foo File "", line 1 pip install foo ^ SyntaxError: invalid syntax >>> Why did you get that response? Because pip is not a python statement (python does not have commands); it is an executable program. So you need to use a command prompt or terminal. -- https://mail.python.org/mailman/listinfo/python-list
Are the critiques in "All the things I hate about Python" valid?
This article is written by Nathan Murthy, a staff software engineer at Tesla. The article is found at: https://medium.com/@natemurthy/all-the-things-i-hate-about-python-5c5ff5fda95e Apparently he chose his article title as "click bait". Apparently he does not really hate Python (So he says.). His leader paragraph is: "Python is viewed as a ubiquitous programming language; however, its design limits its potential as a reliable and high performance systems language. Unfortunately, not every developer is aware of its limitations." As I currently do not have the necessary technical knowledge to properly evaluate his claims, I thought I would ask those of you who do. I have neither the knowledge or boB-hours to write a large distributed system code base, but I am curious if Python is truly limited for doing these in the ways he claims. BTW, I am not trying to start (another) heated, emotional thread. You guys do sometimes get carried away! I honestly just what to know the truth of the matters out of my continuing to learn Python. I suspect there is probably some truth in his claims, but I am not sure if he is taking things out of their proper application contexts or not. Thanks! -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Are the critiques in "All the things I hate about Python" valid?
On Fri, Feb 16, 2018 at 10:05 PM, Ben Finney wrote: > He blithely conflates “weakly typed” (Python objects are not weakly, but > very strongly typed) with “dynamically typed” (yes, Python's name > binding is dynamically typed). Those are two, orthognal dimensions to > describe a language. > > All the assertions about “strongly typed” should instead be read as > “statically typed”. I do not think the author understands the > differences, nor the trade-offs involved, in the weak–strong dimension > nor the static–dynamic dimension. I should have caught the "weakly typed" error. But I was too distracted by his "type safety" train of thought. And this is one I am still puzzling over: Are statically-typed languages inherently "safer" than properly implemented dynamically-typed languages? I can see the advantages of catching type errors at compile time versus run time. And with the relatively recent introduction of type hints to python, I was wondering if this was indeed a perceived, need-to-correct weakness by the python devs? But I have seen myself the advantages of being able to rebind identifiers to differently typed objects than what they started out being bound to. So I do not have enough knowledge to properly evaluate software safety or robustness in terms of how a language is typed. > A strong hint is in the complaint about the Global Interpreter Lock. > This is a property of only one implementation: the CPython > implementation. It is a known limitation and many people have worked for > many years to ameliorate it; it is unclear whether the author knows of > anything done in recent years (the only specific information cited is > for 2012; a reminder, that's *six years ago*, a aeon in platform > development). This is another of my big technical weak spots. I currently believe that doing concurrent and asynchronous programming is a difficult chore in any language, but I have yet gotten to the point of where I know enough to start struggling myself with these forms of programming. I do know from some of my books that there are ways despite the GIL to handle these types of programming problems with the current CPython implementation. But I do not know enough to evaluate how well or poorly python can make use of multicore processors. > The author also uses very old information when complaining about Python > 2 versus Python 3. The citation of grave fears is to a 2014 article. The > author shows no signs of awareness that the Python 3 ecosystem is very > healthy today compared to Python 2. I mostly dismissed out of hand this section. The only disturbing part, in my mind, was the claim that individual companies have python 2 vs. 3 rifts within their companies. I hope that is not really true. > The complaint about abstractions makes another silly conflation; the > author seems to think any language that does object-orientation > differently from Java, is obviously wrong. The author may benefit from > reading http://dirtsimple.org/2004/12/python-is-not-java.html> > (advice which is no less relevant than in 2004). But maybe not. Again, my only concern here was software safety issue claims. Does locking down object attribute access by the language necessarily improve software safety or robustness? I actually like python's "we're all consenting adults" mentality. It seems to me that the quality of the code is more reflective of the quality of the programmers than protecting (mediocre?) programmers from themselves by language design. But then again, maybe these sorts of language design restrictions make for fewer errors, even by excellent programmers, than not doing so. Again, I don't believe I know enough to intelligently evaluate the pros and cons. > The author acknowledges in the conclusion that “nothing I’ve said is > new”. I would argue that most of what is said is not even true; the > article should not be referenced at all, IMO. > Ben, I guess I am trying to be more charitable in my interpretation of the author's intent and actual knowledge. I was led to this article via one of the two weekly python news collation email services that I subscribe to, so I am hoping to learn something from the article and everyone's feedback. What's the old saw? Even a blind squirrel finds a nut once in a while? Thanks! -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Are the critiques in "All the things I hate about Python" valid?
On Fri, Feb 16, 2018 at 10:25 PM, Chris Angelico wrote: > > 1) Type safety. > > This is often touted as a necessity for industrial-grade software. It > isn't... Chris, would you mind expanding on this point? What is necessary for industrial-grade, safe, robust software? Do statically-typed languages give any real advantage over dynamically-typed languages in this context? I know much is achieved by how the software is designed and the practices used to implement the design that are independent of the language used. But what language features/components are essential for industrial-grade software? -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Are the critiques in "All the things I hate about Python" valid?
I've just reread everyone's replies and one point you mentioned about the GIL caught my eye ... On Fri, Feb 16, 2018 at 11:16 PM, Chris Angelico wrote: > Asynchronicity and concurrency are hard. Getting your head around a > program that is simultaneously doing two things is inherently tricky. > Anything that simplifies this (such as providing atomic and > thread-safe operations on high-level data structures) will make this > sort of work FAR easier. That's what Python gives you. CPython's > current implementation (with the GIL) is reasonably easy to > conceptualize, and requires very few other locks to make everything > work sanely; all attempts to remove the GIL from CPython have involved > other tradeoffs, usually resulting in significant performance > penalties on single-threaded code. I am curious as to what efforts have been attempted to remove the GIL and what tradeoffs resulted and why? Is there a single article somewhere that collates this information? I fear if I try to Google this I will get a lot of scattered pieces that I will have to wade through to get to what I want to know, where you (or someone else) might be able to point me to a good link. Or kindly summarize yourself the relevant information. Thanks! -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Are the critiques in "All the things I hate about Python" valid?
On Sat, Feb 17, 2018 at 12:54 AM, Chris Angelico wrote: > On Sat, Feb 17, 2018 at 5:25 PM, boB Stepp wrote: >> >> I am curious as to what efforts have been attempted to remove the GIL >> and what tradeoffs resulted and why? Is there a single article >> somewhere that collates this information? I fear if I try to Google >> this I will get a lot of scattered pieces that I will have to wade >> through to get to what I want to know, where you (or someone else) >> might be able to point me to a good link. Or kindly summarize >> yourself the relevant information. >> >> Thanks! > > No, there isn't a single article, at least not that I know of. A good > word to search for is "gilectomy", which brought me to this talk by > Larry Hastings: > > https://www.youtube.com/watch?v=pLqv11ScGsQ Thanks for the link. I'll give it a look tomorrow. Must get some sleep soon! Meanwhile I just finished reading "Efficiently Exploiting Multiple Cores with Python" (from Nick Coghlan's Python Notes) at http://python-notes.curiousefficiency.org/en/latest/python3/multicore_python.html It answered some of my questions and has a lot of good information. It appears that a "gilectomy" as you put it is a really tough problem. It looks to me that it might require a major backwards-compatibility-breaking change to CPython to implement if I am even close to understanding the issues involved. > Broadly speaking, what happens is that removing a large-scale lock > (the GIL) requires using a whole lot of small-scale locks. That gives > finer granularity, but it also adds a whole lot of overhead; the CPU > features required for implementing those locks are not fast. With the > GIL, you claim it, and you can do what you like. Without the GIL, you > have to claim a lock on each object you manipulate, or something along > those lines. (Different attempts have gone for different forms of > granularity, so I can't generalize too much here.) That means claiming > and relinquishing a lot more locks, which in turn means a lot more > CPU-level "lock" primitives. That's a lot of overhead. Thanks for this explanation. It really helps me a lot! > One of the best ways to multi-thread CPU-intensive work is to push a > lot of the work into an extension library. Take a function like this: > > def frobnosticate(stuff): > magic magic magic > magic magic more magic > return result > > As long as the returned object is a newly-created one and the > parameter is not mutated in any way, you can do all the work in the > middle without holding the GIL. That can't be done in pure Python, but > in a C function, it certainly can. You still have the coarse-grained > locking of the GIL, you still have all the protection, but you can now > have two threads frobnosticating different stuff at the same time. Yeah, Coghlan's article mentioned this. I guess using Cython would be one approach to this. Thanks, Chris! -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: How to make Python run as fast (or faster) than Julia
On Fri, Feb 23, 2018 at 1:16 PM, Chris Angelico wrote: > On Sat, Feb 24, 2018 at 6:09 AM, Python wrote: >> On Sat, Feb 24, 2018 at 05:56:25AM +1100, Chris Angelico wrote: >>> No, not satisfied. Everything you've said would still be satisfied if >>> all versions of the benchmark used the same non-recursive algorithm. >>> There's nothing here that says it's testing recursion, just that (for >>> consistency) it's testing the same algorithm. There is no reason to >>> specifically test *recursion*, unless that actually aligns with what >>> you're doing. >> >> It seems abundantly clear to me that testing recursion is the point of >> writing a benchmark implementing recursion (and very little of >> anything else). Again, you can decide for yourself the suitability of >> the benchmark, but I don't think you can really claim it doesn't >> effectively test what it means to. > > Where do you get that it's specifically a recursion benchmark though? > I can't find it anywhere in the explanatory text. I hope I am not missing something blatantly obvious, but in the page Python linked to (https://julialang.org/benchmarks/), in the opening paragraph it states: These micro-benchmarks, while not comprehensive, do test compiler performance on a range of common code patterns, such as function calls, string parsing, sorting, numerical loops, random number generation, recursion, and array operations. Recursion is listed above as one code pattern to specifically target with one of their micro-benchmarks. I did not exhaustively go through each language's code examples, but it does appear that the authors of these benchmarks are implementing as exactly as possible the same algorithm chosen for each of these code patterns in each language. Now to me the real question is whether the Julia people were trying to be intellectually honest in their choices of coding patterns and algorithms or were they deliberately cherry picking these to make Julia look better than the competition? Myself, I would rather be charitable than accusatory about the benchmarkers' intentions. For instance, the authors were aware of numpy and used it for some of the python coding -- the array operations they were targeting IIRC. Instead, if they were being deliberately dishonest, they could have come up with some really contrived python code. -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Application window geometry specifier
On Wed, Jan 13, 2021 at 7:28 PM Chris Angelico wrote: > I love how "I think" is allowed to trump decades of usability research. Can you recommend a good reference for someone relatively new to GUI programming that is based on such research? Book or web reference would be fine. Cheers! boB -- https://mail.python.org/mailman/listinfo/python-list
Re: IDE tools to debug in Python?
On Sat, Feb 06, 2021 at 01:27:34AM +, flaskee via Python-list wrote: On Friday, February 5, 2021 5:03 PM, Schachner, Joseph wrote: As regards the Alan pdb note; yes -- I was trying for a full IDE & integrated debugging, along the lines of Visual Studio, pre-sneaky-Telemetry API. You do know that there is a 100% open source version of Microsoft VS Code with absolutely no tracking? It is VSCodium: https://vscodium.com/ Perhaps it is more along the lines of what you are looking for? -- Wishing you only the best, boB Stepp -- https://mail.python.org/mailman/listinfo/python-list
Re: New Python implementation
On 21/02/16 11:03AM, Alan Gauld wrote: Python v1 was a good teaching language. v2 complicated it a bit but it was still usable. v3 is no longer a good teaching language (unless maybe you are teaching CompSci at university.) [...] And that's just one example, the language is now full of meta goodness that makes it incomprehensible to beginners. In a teaching environment, with a teacher present to answer questions, it's probably usable, but for a self guided course it's increasingly inscrutable. Hmm. I'm not sure I can agree, Alan. My son took to Python 3 like a duck to water. Occasionally he has had questions for me, but, for the most part, he has been entirely on his own with no issues. He now has several projects that others are using, including a significant update to someone else's project he was interested online to both update the Python and Qt. Yes, there is a ton of meta-goodness in Python, but most of it can be ignored to be able to *just use it*. My wife is using Python 3 in a robotics class she teaches and her high school students find it accessible. AP computer science second semester uses Python. I really don't think it is much of a problem for most people. Of course on the Tutor list there often seems to be some greatly puzzled people, but I think their issues stem from *never* having tried to do anything even remotely technical other than clicking around in the GUI programs they have become comfortable with. -- Wishing you only the best, boB Stepp -- https://mail.python.org/mailman/listinfo/python-list
Re: If you have Python Cookbook, 3rd ed.
On Thu, May 20, 2021 at 11:43 AM Terry Reedy wrote: > > can you verify that the Algorithm chapter (near end in 2nd ed.) does > *NOT* have an introduction by Tim Peters? > (Info needed to verify timeit doc correction > https://github.com/python/cpython/pull/21744) In my 3rd edition copy chapter 1 is "Data Structures and Algorithms". The only thing comprising an introduction is a brief paragraph that starts the chapter and has no attribution. HTH, boB Stepp -- https://mail.python.org/mailman/listinfo/python-list
Re: If you have Python Cookbook, 3rd ed.
On Thu, May 20, 2021 at 3:48 PM Terry Reedy wrote: > > On 5/20/2021 1:14 PM, boB Stepp wrote: > > On Thu, May 20, 2021 at 11:43 AM Terry Reedy wrote: > >> > >> can you verify that the Algorithm chapter (near end in 2nd ed.) does > >> *NOT* have an introduction by Tim Peters? > >> (Info needed to verify timeit doc correction > >> https://github.com/python/cpython/pull/21744) > > > > In my 3rd edition copy chapter 1 is "Data Structures and Algorithms". > > The only thing comprising an introduction is a brief paragraph that > > starts the chapter and has no attribution. > > Hmm. 2nd ed starts with 1. Text. The intro mentioned was for 18. > Algorithms, starting with removing duplicates from a sequence. 3rd edition in chapter 1 starts with "1.1 Unpacking a Sequence into Separate Variables" Later in chapter is "1.10 Removing Duplicates from a Sequence while Maintaining Order". The preface does say the third edition is a major rewrite, so I suppose there will be significant differences from the 2nd edition which I don't own. boB Stepp -- https://mail.python.org/mailman/listinfo/python-list
Re: Turtle module
On Wed, May 26, 2021 at 10:59 AM Michael F. Stemper wrote: > In order to turn the turtle, I need to select a way to represent > angles. I could use either degrees or radians (or, I suppose, > grads). However, for my functions to work, I need to set the > turtle to that mode. This means that I could possibly mess up > things for the caller. What I would like to do is capture the > angle-representation mode on entry and restore it on return. > However, looking through the methods of turtle.Turtle(), I > can't find any means of capturing this information In the "Tell Turtle's state" section (https://docs.python.org/3/library/turtle.html#tell-turtle-s-state) I would think that you could roll your own function by using your knowledge of the coordinates of where the turtle is at and the turtle.towards(0, 0). By computing your own angle (knowing what units you are using) and comparing with what is returned you should be able to determine what angular units are currently set. HTH! boB Stepp -- https://mail.python.org/mailman/listinfo/python-list
Re: imaplib: is this really so unwieldy?
On Thu, May 27, 2021 at 6:22 PM Cameron Simpson wrote: > > On 27May2021 18:42, hw wrote: > >So it seems that IMAP support through python is virtually non-existent. > > This still sureprises me, but I've not tried to use IMAP seriously. I > read email locally, and collect it with POP instead. With a tool I wrote > myself in Python, as it happens. I am out of my league here, but what I found in one of my books might be helpful. Al Sweigart wrote a useful book, "Automate the Boring Stuff in Python". In chapter 16 he considers email. In the "IMAP" section he states: Just as SMTP is the protocol for sending email, the Internet Message Access Protocol (IMAP) specifies how to communicate with an email provider’s server to retrieve emails sent to your email address. Python comes with an imaplib module, but in fact the third-party imapclient module is easier to use. This chapter provides an introduction to using IMAPClient; the full documentation is at http://imapclient.readthedocs.org/. The imapclient module downloads emails from an IMAP server in a rather complicated format. Most likely, you’ll want to convert them from this format into simple string values. The pyzmail module does the hard job of parsing these email messages for you. You can find the complete documentation for PyzMail at http://www.magiksys.net/pyzmail/. Install imapclient and pyzmail from a Terminal window. Appendix A has steps on how to install third-party modules. In the next little section he shows how to retrieve and delete emails with IMAP using the two third-party tools mentioned above. And of course there is more. Apparently this book is now in its second edition. The first edition is available online for free. The link to chapter 16 which discusses email is: https://automatetheboringstuff.com/chapter16/ Hopefully this will prove helpful to the OP. HTH! boB Stepp -- https://mail.python.org/mailman/listinfo/python-list
Re: imaplib: is this really so unwieldy?
On Sun, May 30, 2021 at 1:04 AM hw wrote: > > On 5/28/21 2:36 AM, boB Stepp wrote: > > > > Just as SMTP is the protocol for sending email, the Internet Message > > Access Protocol (IMAP) specifies how to communicate with an email > > provider’s server to retrieve emails sent to your email address. > > Python comes with an imaplib module, but in fact the third-party > > imapclient module is easier to use. This chapter provides an > > introduction to using IMAPClient; the full documentation is at > > http://imapclient.readthedocs.org/. > > > > The imapclient module downloads emails from an IMAP server in a rather > > complicated format. Most likely, you’ll want to convert them from this > > format into simple string values. The pyzmail module does the hard job > > of parsing these email messages for you. You can find the complete > > documentation for PyzMail at http://www.magiksys.net/pyzmail/. > > > > Install imapclient and pyzmail from a Terminal window. Appendix A has > > steps on how to install third-party modules. > > > I don't know which imaplib the author uses; the imaplib I found > definitely doesn't give uids of the messages, contrary to the example > he's giving. Look at the above three paragraphs quoted from my original response. The author is using *imapclient* and *pyzmail* as the author indicates. boB Stepp -- https://mail.python.org/mailman/listinfo/python-list
