Re: Spaces in path name
On Mar 14, 8:15 pm, "David S" <[EMAIL PROTECTED]> wrote: > By mapping network drives in windows I can get past these issues with path > names. > > Thanks, > David > > "Tim Golden" <[EMAIL PROTECTED]> wrote in message > > news:[EMAIL PROTECTED] > > > David S wrote: > >> Gets me further but still seems to be issue with space after 'Program' as > >> code tries to run 'C:\Program'. Don't understand what is going on here... > > > Slight apologies as I haven't followed this thread closely, but using the > > Acrobat Reader executable, which is, I think, good enough for the > > purposes of illustration: > > > > > import os > > import subprocess > > > filename = r"C:\Program Files\Adobe\Reader 8.0\Reader\AcroRd32.exe" > > doc = r"C:\Program Files\Adobe\Reader 8.0\Resource\ENUtxt.pdf" > > > print os.path.isfile (filename) > > > os.system (filename + " " + doc) > > > os.system ('"%s" "%s"' % (filename, doc)) > > > subprocess.call ([filename, doc]) > > > > > > os.path.isfile succeeds > > os.system (filename) fails as your code does > > os.system ('"%s"' ...) fails even though both strings are requoted > > subprocess.call (filename) succeeds > > > The latter, at least, is because the subprocess module > > has some special-case handling for exactly this situation > > on MS Windows, while os.system doesn't. > > > Now, ultimately, I don't know if this really helps your > > exact situation but it least it should be clear what will > > and what won't work. Conclusion: use subprocess.call if > > you can. > > > TJG I had the same problem recently with subprocess.popen, when there is a space in the executable and a space in an additional argument as in the acrobat example. I finally found a past thread in this news group that explained that you have to use two (2) leading double quotes, and as on the command line all arguments with spaces have to be quoted. All three examples with acrobat work on my WindowsXP in this case (opens Acrobat three times, after closing previous): import os import subprocess filename = r"C:\Program Files\Adobe\Acrobat 7.0\Acrobat\Acrobat.exe" doc = r"C:\Program Files\Adobe\Acrobat 7.0\Help\ENU\Pdfmark.pdf" print os.path.isfile (filename) print 'case 1' os.system ('""' + filename + '" "' + doc + '"') # quotes split up for clarity: (' " " ' + filename + ' " " ' + doc + ' " ') print 'case 2' os.system ('""%s" "%s"' % (filename, doc)) print 'case 3' subprocess.call ([filename, doc]) Josef -- http://mail.python.org/mailman/listinfo/python-list
Re: Spaces in path name
> > http://timgolden.me.uk/python/win32_how_do_i/run-a-command-with-a-spa... Note: this works for subprocess.call but for subprocess.Popen this does not work if there are two arguments in the command line with spaces. Especially, even after trying out many different versions, I never managed to get subprocess.Popen to work, when both the executable and one argument have spaces in the file path. subprocess.Popen requires the same extra double quoting at front or around the entire command as os.system. It took me many hours to find the versions of quoting that work. Explanation by Fredrik Lundh in thread http://groups.google.com/group/comp.lang.python/browse_thread/thread/775ca566af9a70c2/65504296a27b43d5?lnk=gst&q=subprocess+windows+spaces#65504296a27b43d5 Here is a test file checking several cases: '''use winrar to get file information of a rar file testing double quotes for windows file paths with spaces using subprocess.Popen see: http://groups.google.com/group/comp.lang.python/browse_thread/thread/775ca566af9a70c2/65504296a27b43d5?lnk=gst&q=subprocess+windows+spaces#65504296a27b43d5 result: * most robust: use double quotes around entire command string * command as list argument never works when 2 file paths have spaces ''' import sys, os import subprocess def rargetinfo(filename,version=0): '''use winrar to get file information of a rar file ''' if version == 0: rarcmd = '"%s %s %s ' % (r'"C:\Program Files\WinRAR\Rar.exe"', "v", filename) elif version == 1: rarcmd = '"%s %s %s" ' % (r'"C:\Program Files\WinRAR \Rar.exe"', "v", filename) elif version == 2: rarcmd = '%s %s %s ' % (r'"C:\Program Files\WinRAR\Rar.exe"', "v", filename) elif version == 3: rarcmd = [r'"C:\Program Files\WinRAR\Rar.exe"', "v", filename] elif version == 4: executable_name = os.path.join(r'C:\Program Files\WinRAR', 'Rar.exe') rarcmd = [executable_name, "v", filename] p = subprocess.Popen(rarcmd, shell=True, stdout=subprocess.PIPE) rettext,reterror = p.communicate() retcode = p.wait() p.stdout.close() print rarcmd print 'this is rettext',rettext[:50] return rettext, rarcmd filenames = [r'"C:\temp\Copy of papers.rar"'] filenames.append(r'"C:\temp\papers.rar"') filenames.append(r'C:\temp\papers.rar') positives = {} negatives = {} for filename in filenames: for version in range(5): print '\n%s version: %d' % (filename,version) rettext, cmd = rargetinfo(filename,version) if rettext : print 'success' positives.setdefault(version,[]).append(cmd) else: print 'failure' negatives.setdefault(version,[]).append(cmd) from pprint import pprint print 'positives:' pprint(positives) print 'negatives:' pprint(negatives) -- http://mail.python.org/mailman/listinfo/python-list
Re: Spaces in path name
On Mar 15, 5:42 pm, joep <[EMAIL PROTECTED]> wrote: > >http://timgolden.me.uk/python/win32_how_do_i/run-a-command-with-a-spa... > > Note: this works for subprocess.call but for subprocess.Popen this > does not work if there are two arguments in the command line with > spaces. Especially, even after trying out many different versions, I > never managed to get subprocess.Popen to work, when both the > executable and one argument have spaces in the file path. > Sorry, incomplete sentence Especially, even after trying out many different versions, I never managed to get subprocess.Popen to work '''when command line is given as a *list* argument to subprocess.Popen''' in the case when both the executable and one argument have spaces in the file path. joe -- http://mail.python.org/mailman/listinfo/python-list
Re: replace string in a file
you can also use standard module fileinput.input with ''inplace'' option which backs up original file automatically. from python help for fileinput: Optional in-place filtering: if the keyword argument inplace=1 is passed to input() or to the FileInput constructor, the file is moved to a backup file and standard output is directed to the input file (if a file of the same name as the backup file already exists, it will be replaced silently). -- http://mail.python.org/mailman/listinfo/python-list
Re: Spaces in path name
Tim Golden wrote: > subprocess.call ([ > >r"C:\Program Files\Adobe\Acrobat 5.0\Reader\acro reader.exe", > > r"C:\Program Files\Adobe\Acr > obat 5.0\Reader\plug_ins.donotuse\Annotations\Stamps\abc def.pdf" > > ]) > > Can you confirm that something equivalent *doesn't* work on your > setup? Or have I misunderstood your point earlier? I'd really > like to get to the point where we can definitively state: this > works (and possibly: that doesn't). > > Thanks > TJG This works without problems for me on Windows XP, Python 2.4.3 In the past, I didn't have problems with subprocess.call, and I never had to look closer to the different versions of quoting. I often had problems understanding subprocess.Popen and getting it to work. For example the equivalent case for popen, as your subprocess.call example, does not work, or I'm making a different mistake with subprocess.Popen: p = subprocess.Popen([ r"C:\Program Files\WinRAR\Rar.exe", r"C:\temp\Copy of papers.rar"], shell=True, stdout=subprocess.PIPE) rettext,reterror = p.communicate() retcode = p.wait() p.stdout.close() results in: 'C:\Program' is not recognized as an internal or external command, operable program or batch file. I assume that there is some difference how subprocess.call and subprocess.Popen handle and format the command. subprocess.Popen does the correct formatting when only one file path has spaces and requires double quoting, but not if there are two file paths with spaces in it. Josef -- http://mail.python.org/mailman/listinfo/python-list
Re: Spaces in path name
Tim Golden wrote: > Tim Golden wrote: > > What I haven't investigated yet is whether the additional flags > > your example is passing (shell=True etc.) cause the main Popen > > mechanism to take a different path. > > Sure enough, passing shell=True -- which is probably quite > a rare requirement -- causes the code to change the call > from "a.exe b.doc" to '%COMSPEC% /c "a.exe" "b.doc"'. > The quoting rules (from cmd /?) are slightly involved but > I can't see at first why this shouldn't work. However it > clearly doesn't so I'll try to put together either a patch > to the subprocess code or to the docs warning of the behaviour. > > I think that, in general, you need to pass shell=True far less > often that you might imagine. (Possibly only for internal > commands like dir, copy etc.). > > TJG Thanks, I didn't know it is possible to drop the shell=True. The explanation of the subprocess model are not very clear to me and the examples are quite incomplete. I got it to work by trial and error and looking at the usage in different packages. Dropping shell=True works well for my case, no error messages and it still captures the standard output: p = subprocess.Popen([ r"C:\Program Files\WinRAR\Rar.exe","v", r"C:\temp\Copy of papers.rar"], stdout=subprocess.PIPE) I am still looking at different versions, and the cases when it works and when it doesn't are still confusing. Josef -- http://mail.python.org/mailman/listinfo/python-list
Re: Spaces in path name
One more try (using popen instead of call is not necessary for these cases, but I want to see the behavior of popen): shell=True and executable and at least one argument with spaces does not work: - p = subprocess.Popen([r"C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin \Copy of cp.exe", r"C:\Temp\temp with spaces\test out.txt", r"C:\Temp\temp with spaces\test out3.txt"], shell=True, stdout=subprocess.PIPE) causes: 'C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin\Copy' is not recognized as an int ernal or external command, operable program or batch file. the following all work: without shell=True, and executable and at least one argument with spaces --- p = subprocess.Popen([r"C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin \Copy of cp.exe", r"C:\Temp\temp with spaces\test out.txt", r"C:\Temp\temp with spaces\test out4.txt"], stdout=subprocess.PIPE) with shell=True, and executable without spaces, even if two arguments with spaces p = subprocess.Popen(["copy", r"C:\Temp\temp with spaces\test out.txt", r"C:\Temp\temp with spaces\test out2.txt"], shell=True, stdout=subprocess.PIPE) with shell=True, and executable without spaces, even if two arguments with spaces but here shell=True is not necessary p = subprocess.Popen([r"C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin \cp.exe", r"C:\Temp\temp with spaces\test out.txt", r"C:\Temp\temp with spaces\test out5.txt"], shell=True, stdout=subprocess.PIPE) without shell=True, and executable and at least one arguments with spaces - p = subprocess.Popen([r"C:\Programs\GnuWin32\GetGnuWin32\gnuwin32\bin \Copy of cp.exe", r"C:\Temp\temp with spaces\test out.txt", r"C:\Temp\temp with spaces\test out4.txt"], stdout=subprocess.PIPE) My conclusions: Assuming shell=True is only required for build-in shell commands, non of which has spaces. There is no problem, if you know when *not* to use shell=True: * More than two arguments with spaces are never problem, as long as the executable does not have spaces * If shell=True is required, then the executable is a build-in shell command, which does not contain spaces, and, therefore, has no problems * If you use a non-build in executable, then don't use shell=True. This works correctly even if the executable and at least one additional argument have spaces. It took a lot of time to figure this out, but now at least I know, how to construct the call to subprocess.Popen, so that it works in the cases I used it so far. Josef -- http://mail.python.org/mailman/listinfo/python-list
Re: replace string in a file
On Mar 16, 10:35 pm, sturlamolden <[EMAIL PROTECTED]> wrote: > On 15 Mar, 21:54, Unknown <[EMAIL PROTECTED]> wrote: > > > I was expecting to replace the old value (serial) with the new one > > (todayVal). Instead, this code *adds* another line below the one found... > > > How can I just replace it? > > A file is a stream of bytes, not a list of lines. You can't just > replace a line with another, unless they have the exact same length. > You must rewrite the whole file to get it right. An example: looks for all 'junk*.txt' files in current directory and replaces in each line the string 'old' by the string 'new' import os, glob, fileinput allfiles = glob.glob(os.getcwd() + '\\junk*.txt') # makes absolute paths findstr = 'old' replstr = 'new' countlinesfound = 0 for line in fileinput.input(allfiles,inplace=1): if line.find(findstr) != -1: line = line.replace(findstr,replstr) # old string , new string countlinesfound += 1 print line,# this writes line back to the file print countlinesfound I found something similar in a tutorial when I started to learn Python, but I don't remember which one. Josef -- http://mail.python.org/mailman/listinfo/python-list
strace of python shows nonsense
Hi there, I am puzzled at how I borked my installation. Python loads slow on my machine, and I decided to use strace and /usr/bin/time to see what is actually happening. # sync && echo 3 > /proc/sys/vm/drop_caches $ /usr/bin/time python2 -c "" 0.19user 0.04system 0:01.22elapsed 19%CPU (0avgtext+0avgdata 4200maxresident)k 7312inputs+0outputs (4major+1145minor)pagefaults 0swaps And when all is in memory: $ /usr/bin/time python2 -c "" 0.19user 0.01system 0:00.21elapsed 98%CPU (0avgtext+0avgdata 4216maxresident)k 0inputs+0outputs (0major+1153minor)pagefaults 0swaps $ /usr/bin/time python2 -c "import argparse" 0.36user 0.02system 0:00.39elapsed 98%CPU (0avgtext+0avgdata 5752maxresident)k 0inputs+0outputs (0major+1699minor)pagefaults 0swaps .2 and .4 seconds to not even get started when all disk I/O is cached. So what is happening here? $ strace -c python2 -c "" % time seconds usecs/call callserrors syscall -- --- --- - - 93.260.001910 8 230 168 open 3.660.75 9 8 mprotect 3.080.63 197 fstat64 0.000.00 0 172 read 0.000.00 063 close <...> $ strace -c python2 -c "import argparse" % time seconds usecs/call callserrors syscall -- --- --- - - 51.890.003732 13 290 read 47.290.003401 18 192 155 stat64 0.820.59 0 129 mmap2 0.000.00 0 549 443 open 0.000.00 0 107 close <...> What puzzles me, is the amount of errors for open and stat64. There are references to stuff I don't know (~/GNUStep directory? Stuff under site-packages that does not exist? Subdirs of site-packages that are not included in sys.path?) What is python doing there, and why? And, more importantly, how can this be corrected? Probably irrelevant, but Python2 version 2.7.3, Archlinux (current as of previous weekend). Thanks, Joep -- http://mail.python.org/mailman/listinfo/python-list
Multiple python versions, one dev environment???
Hello! The condensed version of the question would probably be: How does one deal with multiple interpreters and one package where you want to try some changes? The background: I made a trivial change to some package (docutils) to scratch a personal itch, and I want to offer this back to the community (whether it will be accepted or not). Because of this, I ran into some issues. Some facts: 1. Python3 is my main interpreter. 2. The tests of docutils only run under python2. 3. I desire not to touch my distribution's version of site-packagesX-Y. 4. I prefer to have one and only one development directory of my target package. My confusions: 1. Is it possible to have one source control managed directory of some package, which is used by two versions of python? 2. I assume that the *.pyc-files generated while using some python source are version dependent. What is the recommended way to have 'em both installed? 3. The way I have stumped a wall over here, is the puzzle of how to make python2 have a different $PYTHONPATH as python3. I hope to hear how this strategy is silly :) 4. I have contemplated the way of linking the source files from my development directory into user specified site-packages directories. Problem 3. still is valid. 5. Should venv and friends/foes com into play? If so: How? Appreciate any light shed on these issues. Thanks! Joep -- https://mail.python.org/mailman/listinfo/python-list
Re: Multiple python versions, one dev environment???
Hello Javier! Thanks, those links are helping a bit. And: yes, I am using Archlinux. But still those links assume that there are totally separate site-packages* directories installed for both. I am not sure how I would surpass this distinction between py-X.P and py-Y.Q. Should I really create a massive amount of symlinks like this?: | #!/usr/bin/zsh | for d in ~/src/mypackage/**/*(/); do |# matches all directories |mkdir -p "~/src/py-2.7/mypackage/${d#*src/mypackage}" |mkdir -p "~/src/py-3.4/mypackage/${d#*src/mypackage}" | done | for f in ~/src/mypackage/**/^*.pyc(.); do |# matches all files except for *.pyc |ln -s "$f" "~/src/py-2.7/mypackage${f#*src/mypackage}" |ln -s "$f" "~/src/py-3.4/mypackage${f#*src/mypackage}" | done ...and then set $PYTHONPATH according to the target version in a #!/usr/local/bin-script? I can work with this (have not tried though), but there must be a more elegant solution than symlinking my way forward... Cheers! Joep On Thu, 17 Jul 2014 16:05:27 + (UTC) Javier wrote: > Are you using arch linux. > > > I deal with multiple interpreters putting fake executables in > /usr/local/bin for everything: (python, sphinx, virtualenv, pydoc, > idle, python-config...) selecting 2 or 3. You can do the same for > selecting 2.3, 2.5, 2.7. What the scripts do is to detect whether > it is a system script whose prefix starts with /usr/bin, or whether > it is a user script. Being in /usr/local/bin they will override > executables in /usr/bin. Remember to chmod a+x the files > in /usr/local/bin > > http://sindhus.bitbucket.org/manage-python-2-3.html > http://stackoverflow.com/questions/15400985/how-to-completely-replace-python-3-with-python-2-in-arch-linux > https://wiki.archlinux.org/index.php/Python#Dealing_with_version_problem_in_build_scripts > > I use these scripts, but there is more than one way to do it > > /usr/local/bin/python=== > #!/bin/bash > script=`readlink -f -- "$1"` > case "$script" in > /usr/bin*) > exec python3 "$@" > ;; > esac > exec python2 "$@" > > /usr/local/bin/virtualenv=== > #!/bin/bash > script=`readlink -f -- "$1"` > case "$script" in > /usr/bin*) > exec virtualenv3 "$@" > ;; > esac > > exec virtualenv2 "$@" > > > > > > > > > Joep van Delft wrote: > > Hello! > > > > The condensed version of the question would probably be: How does > > one deal with multiple interpreters and one package where you > > want to try some changes? > > > > The background: > > I made a trivial change to some package (docutils) to scratch a > > personal itch, and I want to offer this back to the community > > (whether it will be accepted or not). Because of this, I ran into > > some issues. > > > > Some facts: > > 1. Python3 is my main interpreter. > > 2. The tests of docutils only run under python2. > > 3. I desire not to touch my distribution's version of > > site-packagesX-Y. > > 4. I prefer to have one and only one development directory of > > my target package. > > > > My confusions: > > 1. Is it possible to have one source control managed directory of > > some package, which is used by two versions of python? > > 2. I assume that the *.pyc-files generated while using some python > > source are version dependent. What is the recommended way to > > have 'em both installed? > > 3. The way I have stumped a wall over here, is the puzzle of how > > to make python2 have a different $PYTHONPATH as python3. I hope to > > hear how this strategy is silly :) > > 4. I have contemplated the way of linking the source files from my > > development directory into user specified site-packages > > directories. Problem 3. still is valid. > > 5. Should venv and friends/foes com into play? If so: How? > > > > Appreciate any light shed on these issues. > > > > Thanks! > > > > > >Joep > > > > > > -- https://mail.python.org/mailman/listinfo/python-list
Re: Multiple python versions, one dev environment???
On Thu, 17 Jul 2014 15:41:44 -0400 Ned Batchelder wrote: > > For testing one project on multiple versions of Python, use tox. > Its entire reason for being is to test Python code against multiple > environments, generally for different Python versions, but possibly > for other reasons, like different versions of dependencies. > > Tox will manage the virtualenvs for you, it makes multi-version > testing very simple. > Excellent, Ned and Akira, I will look into it! Cheers, Joep -- https://mail.python.org/mailman/listinfo/python-list