piping with subprocess
I spent half a day trying to convert this bash script (on Mac) textutil -convert html $1 -stdout | pandoc -f html -t markdown -o $2 into Python using subprocess pipes. It works if I save the above into a shell script called convert.sh and then do subprocess.check_call(["convert.sh", file, markdown_file]) where file and markdown_file are variables. But otherwise my piping attempts fail. Could someone show me how to pipe in subprocess. Yes, I've read the doc, especially http://docs.python.org/2/library/subprocess.html#replacing-shell-pipeline But I'm a feeble hobbyist, not a computer scientist. Thanks RD -- https://mail.python.org/mailman/listinfo/python-list
Re: piping with subprocess
On Saturday, February 1, 2014 6:54:09 AM UTC-6, Peter Otten wrote: > Rick Dooling wrote: > > > > > I spent half a day trying to convert this bash script (on Mac) > > > > > > textutil -convert html $1 -stdout | pandoc -f html -t markdown -o $2 > > > > > > into Python using subprocess pipes. > > > > > > It works if I save the above into a shell script called convert.sh and > > > then do > > > > > > subprocess.check_call(["convert.sh", file, markdown_file]) > > > > > > where file and markdown_file are variables. > > > > > > But otherwise my piping attempts fail. > > > > It is always a good idea to post your "best effort" failed attempt, if only > > to give us an idea of your level of expertise. > > > > > Could someone show me how to pipe in subprocess. Yes, I've read the doc, > > > especially > > > > > > http://docs.python.org/2/library/subprocess.html#replacing-shell-pipeline > > > > > > But I'm a feeble hobbyist, not a computer scientist. > > > > Try to convert the example from the above page > > > > """ > > output=`dmesg | grep hda` > > # becomes > > p1 = Popen(["dmesg"], stdout=PIPE) > > p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) > > p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. > > output = p2.communicate()[0] > > """ > > > > to your usecase. Namely, replace > > > > ["dmesg"] --> ["textutil", "-convert", "html", infile, "-stdout"] > > ["grep", "hda"] --> ["pandoc", "-f", "html", "-t", "marktown", "-o", > > outfile] > > > > Don't forget to set > > > > infile = ... > > outfile = ... > > > > to filenames (with absolute paths, to avoid one source of error). > > If that doesn't work post the code you wrote along with the error messages. p1 = subprocess.Popen(["textutil", "-convert", "html", file], stdout=subprocess.PIPE) p2 = subprocess.check_call(["pandoc", "-f", "html", "-t", "markdown", "-o", markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE) p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. output = p2.communicate()[0] Errors Traceback (most recent call last): File "/Users/me/Python/any2pandoc.py", line 70, in convert_word_file(file, markdown_file) File "/Users/me/Python/any2pandoc.py", line 59, in convert_word_file output = p2.communicate()[0] AttributeError: 'int' object has no attribute 'communicate' I get a markdown_file created but it's empty. Thanks, RD ps - Daniel's works fine but I still don't learn to pipe :) -- https://mail.python.org/mailman/listinfo/python-list
Re: piping with subprocess
On Saturday, February 1, 2014 7:54:34 AM UTC-6, Rick Dooling wrote: > On Saturday, February 1, 2014 6:54:09 AM UTC-6, Peter Otten wrote: > > > Rick Dooling wrote: > > > > > > > > > > > > > I spent half a day trying to convert this bash script (on Mac) > > > > > > > > > > > > > > textutil -convert html $1 -stdout | pandoc -f html -t markdown -o $2 > > > > > > > > > > > > > > into Python using subprocess pipes. > > > > > > > > > > > > > > It works if I save the above into a shell script called convert.sh and > > > > > > > then do > > > > > > > > > > > > > > subprocess.check_call(["convert.sh", file, markdown_file]) > > > > > > > > > > > > > > where file and markdown_file are variables. > > > > > > > > > > > > > > But otherwise my piping attempts fail. > > > > > > > > > > > > It is always a good idea to post your "best effort" failed attempt, if only > > > > > > to give us an idea of your level of expertise. > > > > > > > > > > > > > Could someone show me how to pipe in subprocess. Yes, I've read the doc, > > > > > > > especially > > > > > > > > > > > > > > http://docs.python.org/2/library/subprocess.html#replacing-shell-pipeline > > > > > > > > > > > > > > But I'm a feeble hobbyist, not a computer scientist. > > > > > > > > > > > > Try to convert the example from the above page > > > > > > > > > > > > """ > > > > > > output=`dmesg | grep hda` > > > > > > # becomes > > > > > > p1 = Popen(["dmesg"], stdout=PIPE) > > > > > > p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) > > > > > > p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. > > > > > > output = p2.communicate()[0] > > > > > > """ > > > > > > > > > > > > to your usecase. Namely, replace > > > > > > > > > > > > ["dmesg"] --> ["textutil", "-convert", "html", infile, "-stdout"] > > > > > > ["grep", "hda"] --> ["pandoc", "-f", "html", "-t", "marktown", "-o", > > > > > > outfile] > > > > > > > > > > > > Don't forget to set > > > > > > > > > > > > infile = ... > > > > > > outfile = ... > > > > > > > > > > > > to filenames (with absolute paths, to avoid one source of error). > > > > > > If that doesn't work post the code you wrote along with the error messages. > > > > p1 = subprocess.Popen(["textutil", "-convert", "html", file], > stdout=subprocess.PIPE) > > p2 = subprocess.check_call(["pandoc", "-f", "html", "-t", "markdown", "-o", > markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE) > > p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. > > output = p2.communicate()[0] > > > > Errors > > > > Traceback (most recent call last): > > File "/Users/me/Python/any2pandoc.py", line 70, in > > convert_word_file(file, markdown_file) > > File "/Users/me/Python/any2pandoc.py", line 59, in convert_word_file > > output = p2.communicate()[0] > > AttributeError: 'int' object has no attribute 'communicate' > > > > I get a markdown_file created but it's empty. > > > > Thanks, > > > > RD > > > > ps - Daniel's works fine but I still don't learn to pipe :) Okay, sorry. I fixed that obvious goof p1 = subprocess.Popen(["textutil", "-convert", "html", file], stdout=subprocess.PIPE) p2 = subprocess.Popen(["pandoc", "-f", "html", "-t", "markdown", "-o", markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE) p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. output = p2.communicate()[0] Now I get no errors, but I still get a blank markdown file. -- https://mail.python.org/mailman/listinfo/python-list
Re: piping with subprocess
On Saturday, February 1, 2014 8:00:59 AM UTC-6, Rick Dooling wrote: > On Saturday, February 1, 2014 7:54:34 AM UTC-6, Rick Dooling wrote: > > > On Saturday, February 1, 2014 6:54:09 AM UTC-6, Peter Otten wrote: > > > > Rick Dooling wrote: > > > > > I spent half a day trying to convert this bash script (on Mac) > > > > textutil -convert html $1 -stdout | pandoc -f html -t markdown -o $2 > > > > into Python using subprocess pipes. > > > > > It works if I save the above into a shell script called convert.sh and > > > > then do > > > > > > subprocess.check_call(["convert.sh", file, markdown_file]) > > > > > where file and markdown_file are variables. > > > > > But otherwise my piping attempts fail. > > > > It is always a good idea to post your "best effort" failed attempt, if > > > only to give us an idea of your level of expertise. > > > > Could someone show me how to pipe in subprocess. Yes, I've read the doc, > > > > especially > > > > http://docs.python.org/2/library/subprocess.html#replacing-shell-pipeline > > > > > But I'm a feeble hobbyist, not a computer scientist. > > > > Try to convert the example from the above page > > > > """ > > > > output=`dmesg | grep hda` > > > > # becomes > > > > p1 = Popen(["dmesg"], stdout=PIPE) > > > > p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE) > > > > p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. > > > > output = p2.communicate()[0] > > > > """ > > > > to your usecase. Namely, replace > > > > ["dmesg"] --> ["textutil", "-convert", "html", infile, "-stdout"] > > > > ["grep", "hda"] --> ["pandoc", "-f", "html", "-t", "marktown", "-o" > > > outfile] > > > > Don't forget to set > > > > infile = ... > > > > outfile = ... > > > > to filenames (with absolute paths, to avoid one source of error). > > > > If that doesn't work post the code you wrote along with the error > > > messages. > > > p1 = subprocess.Popen(["textutil", "-convert", "html", file], > > stdout=subprocess.PIPE) > > > p2 = subprocess.check_call(["pandoc", "-f", "html", "-t", "markdown", "-o", > > markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE) > > > p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. > > output = p2.communicate()[0] > > Errors > > Traceback (most recent call last): > > File "/Users/me/Python/any2pandoc.py", line 70, in > > convert_word_file(file, markdown_file) > > File "/Users/me/Python/any2pandoc.py", line 59, in convert_word_file > > output = p2.communicate()[0] > > AttributeError: 'int' object has no attribute 'communicate' > > I get a markdown_file created but it's empty. > > Thanks, > > RD > > ps - Daniel's works fine but I still don't learn to pipe :) > Okay, sorry. I fixed that obvious goof > p1 = subprocess.Popen(["textutil", "-convert", "html", file], > stdout=subprocess.PIPE) > > p2 = subprocess.Popen(["pandoc", "-f", "html", "-t", "markdown", "-o", > markdown_file], stdin=p1.stdout, stdout=subprocess.PIPE) > > p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. > > output = p2.communicate()[0] > > Now I get no errors, but I still get a blank markdown file. Okay, blank lines removed. Apologies. I didn't know Google inserted them. RD -- https://mail.python.org/mailman/listinfo/python-list
checking whether a string is text or binary
I'm an English major who hacks scripts together to do things as needed.
I used this code from the Python Cookbook for years.
https://www.safaribooksonline.com/library/view/python-cookbook-2nd/0596007973/ch01s12.html
Especially when I need to convert old WPD files to markdown, some of which
don't even have file extensions, so the code helps determine what kind of file
it is I am reading in conversion scripts.
I tried to convert the code using 2to3 and it broke.
The error I get when using Python 3 is on this line:
_null_trans = string.maketrans("", "")
and the error reads
AttributeError: 'module' object has no attribute 'maketrans'
Any help? I barely understand the whole unicode business, but any guidance in
updating the code would be much appreciated.
THANKS
Rick
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python on a MacBook Pro (not my machine)
Just upgraded to Mavericks, the new OS X, Python is: Python 2.7.5 (default, Aug 25 2013, 00:04:04) [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> (On Mountain Lion, it was 2.7.2. You can install Python 3 using Home Brew or even the packages from Python.org and run them on the same machine. Just change the shebang at the to of the script. I recommend MacVim for editing http://code.google.com/p/macvim/ On Saturday, October 26, 2013 2:07:40 PM UTC-5, John Ladasky wrote: > Hi folks, > > > > My side job as a Python tutor continues to grow. In two weeks, I will start > working with a high-school student who owns a MacBook Pro. > > > > I have had students with Linux systems (my preference) and Windows systems > before, but not Macs. On my first visit, I set up each student's computer > with Python 3.x, and SciTE for editing. I would like to do something similar > for my Mac student, and I want to make sure that it goes smoothly. > > > > My first question is whether Mac OS X ships with Python 2.x, and whether I > need to be aware of any compatibility issues when I install 3.x. (It's 2013, > and my students are new to programming. I refuse to hitch them to Python 2.) > > > > Second: it doesn't look like I will be able to obtain SciTE for this student. > SciTE is free for Windows and Linux. Apparently, it's $42 for Mac OSX? If > I recall, SciTE is open-source, so I suppose that I could compile the source > myself. But since it is not my computer, and I'm being paid for my time, and > I haven't done much with Macs (to say nothing of building from source code), > I don't think that this is appropriate. > > > > I know, we can use IDLE. I continue to find IDLE clumsy. Also, there are > potential issues with event handling which arise when you use IDLE. I am > working with an adult professional who is developing a Telnet application, > which refuses to cooperate with IDLE/Tk. I had similar issues myself with > wxPython applications I was writing. While these issues may not affect a > beginning student, these experiences have informed my choices. > > > > So, what other free and lightweight editing options do I have for a Mac? I > have found a few (fairly old) discussions on comp.lang.python which suggest > Eric (http://eric-ide.python-projects.org/) and Editra (http://editra.org/). > Opinions on these and other choices are appreciated. > > > > Thanks! -- https://mail.python.org/mailman/listinfo/python-list
Any other screenwriters?
Hello all, I am an amateur Python person, and I usually learn just enough to make one writing tool or another as I go, because mainly I'm a writer, not a programmer. Recently, I've been exploring a markdown syntax called Fountain for screenwriters http://fountain.io/syntax https://github.com/nyousefi/Fountain There are several apps that purportedly will allow screenwriters to convert plain text files to PDFs that follow screenwriting format. They cannot replace the $250 programs we use once production begins, but they are good enough for working drafts until production begins. However, most of these programs are "apps" with closed GUIs, and of course I'm looking for a way to do the same thing with Python and call it from the command-line or from within Vim To that end, I would like to take this Ruby script (which works pretty well, but throws errors in Mac OS X; some Ruby ones and some Prince ones) and convert it to Python so I can fix it myself, because I don't know Ruby at all, and would rather work in Python. https://github.com/olivertaylor/Textplay Any pointers? Thanks a bunch, Rick Dooling -- http://mail.python.org/mailman/listinfo/python-list
Re: importing a user file in Python
On Dec 1, 1:42 am, waltbrad <[EMAIL PROTECTED]> wrote: > Hello. I'm brand new to Python. > > Where on my system do I have to place these files before the > interpreter will import them? > In this case, odbchelper.py is a module you are trying to import. http://docs.python.org/tut/node8.html One suggestion is to make a directory in your /home dir where you keep your downloaded and homemade Python modules and scripts. I call mine "Python" so it's ~/Python or /home/rick/Python. Then, as suggested, add this dir to your Python path, so that you can import your custom modules (with import odbchelper). You may also want to add that dir to your system path, so that you can call your Python scripts from anywhere without having to change back to the ~/Python directory: export PATH=~/Python:~/bin:$PATH Here I've added two directories: Python and bin to $Path so that the system knows to look for executables there. Good luck. Rick -- http://mail.python.org/mailman/listinfo/python-list
Re: python 2.5 - f=open('a_file.txt','w') gives [Errno 2]
On Dec 3, 7:47 am, dirkheld <[EMAIL PROTECTED]> wrote: > IOError: [Errno 2] No such file or directory: 'a_file.txt' I sometimes see that error on Linux when trying to run a script with DOS line endings. Is it an imported file? I don't know Macs, but start by making sure both your script and the file have Mac line endings? rick -- http://mail.python.org/mailman/listinfo/python-list
Re: python 2.5 - f=open('a_file.txt','w') gives [Errno 2]
> Sorry for keeping you guys busy with such a mistake ;) No apologies necessary, especially since you reported the final outcome. Now anybody searching on that message will find a complete thread and a lesson learned. rick -- http://mail.python.org/mailman/listinfo/python-list
Re: newbie
On Dec 10, 8:03 pm, "Whizzer" <[EMAIL PROTECTED]> wrote: > Is OReilly's Learning Python a good place to start learning to program? > I've been told Python is a good first language. > > Thanks for the advice. If you already have Python installed,just go to the bottom of this article and check the various links and recommended books. Good luck. http://tinyurl.com/w7wgp rd -- http://mail.python.org/mailman/listinfo/python-list
Re: finding dir of main .py file
On Dec 11, 10:08 am, "ron.longo" <[EMAIL PROTECTED]> wrote:
> Is there any way that I can find the path of the main .py file of my
> application?
>
> For example, I have an application with some resources which are in a
> subdirectory:
>
> myPythonApp.py
> /resources
> image1
> image2
> etc.
I just put the reference in my module. Don't hard code an absolute
path, use the environment tools.
app_path = os.getenv('HOME') + "/your_sub_dir"
resources_path = os.getenv('HOME') + "/your_sub_dir/resources"
If there's another way, someone else will jump in.
rd
--
http://mail.python.org/mailman/listinfo/python-list
Re: re.sub() problem (regular expression)
On Dec 13, 9:00 pm, Davy <[EMAIL PROTECTED]> wrote: > > What's "\1" and the whole re.sub() mean? > Read about backreferences here: http://www.regular-expressions.info/brackets.html Also see the entry on parentheses here: http://docs.python.org/lib/re-syntax.html rick -- http://mail.python.org/mailman/listinfo/python-list
Re: listdir() with mask
On Dec 14, 1:56 am, "Vladimir Rusinov" <[EMAIL PROTECTED]> wrote: glob or fnmatch http://docs.python.org/lib/module-glob.html rd -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie observations
On Dec 18, 2:14 pm, [EMAIL PROTECTED] wrote: > But where? Look it up in the function reference. OK, > where's the function reference? A line of code that you'd type in a > second is a ten-minute search. Thank God for google. Maybe this will help: http://rgruet.free.fr/PQR25/PQR2.5.html But since you're already a programmer, you really should take an hour and read the tutorial. It was written for people like you. rd -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie question - what's the term for input/output to a web page?
On Dec 21, 7:03 pm, Rachel Garrett <[EMAIL PROTECTED]> wrote: > I'd like to write a simple application that will accept input from the > user, go out to a particular web page, and submit the user's input to > the website. The results that are displayed by the web page should > then be sent back to the application. Try urllib2 module. Here's a mini howto http://www.voidspace.org.uk/python/articles/urllib2.shtml rd -- http://mail.python.org/mailman/listinfo/python-list
Re: Open source English dictionary to use programmatically w/ python
On Jan 7, 4:37 pm, dgoldsmith_89 <[EMAIL PROTECTED]> wrote: > Can anyone point me to a downloadable open source English dictionary > suitable for programmatic use with python: I'm programming a puzzle > generator, and I need to be able to generate more or less complete > lists of English words, alphabetized. Thanks! DG On Linux? WordNet and Dict and many others. On Windows, maybe try WordWeb? rd -- http://mail.python.org/mailman/listinfo/python-list
Re: Python noob SOS (any [former?] Perlheads out there?)
On Jan 29, 10:39 am, kj <[EMAIL PROTECTED]> wrote: > I'd written a Perl module to facilitate the writing of scripts. > It contained all my boilerplate code for parsing and validating > command-line options, generating of accessor functions for these > options, printing of the help message and of the full documentation, > testing, etc. http://docs.python.org/lib/optparse-tutorial.html rd -- http://mail.python.org/mailman/listinfo/python-list
Re: Does __import__ require a module to have a .py suffix?
On Mar 12, 11:22 am, mrstephengross <[EMAIL PROTECTED]> wrote:
> Hi all. I've got a python file called 'foo' (no extension). I want to
> be able to load it as a module, like so:
>
> m = __import__('foo')
>
> However, the interpreter tells me "No module named foo". If I rename
> it foo.py, I can indeed import it. Is the extension required? Is there
> any way to override that requirement?
>
I think you answered your own question, but if you want more info:
>From the Python Tutorial:
http://docs.python.org/tut/node8.html
"A module is a file containing Python definitions and statements. The
file name is the module name with the suffix .py appended."
RD
> Thanks,
> --Steve
--
http://mail.python.org/mailman/listinfo/python-list
Re: Running a python program as main...
On Mar 26, 9:12 am, waltbrad <[EMAIL PROTECTED]> wrote: > On his command line he types: > > C:\...\PP3E>Launcher.py > > and this begins the program. Doesn't work for me. I have to type: > > C:\...\PP3E>python Launcher.py > > Is this a typo on his part or has he configured his settings in such a > way that the command line will automatically associate the extension > with the program? (If so, he didn't mention this in his book). Browse this thread on clp: http://tinyurl.com/2wbnde RD -- http://mail.python.org/mailman/listinfo/python-list
Re: Data Coding suggestions
On Feb 27, 6:42 am, "steven.oldner" wrote: > Just learning Python and have a project to create a weekly menu and a > shopping list from the menu. > Question: How should I set up the data? I'm looking at maybe 70 menu > items and maybe 1000 items for the shopping list. I need to be able > to maintain each item also. > I agree with Mr. Holden. It's a perfect exercise for using Python with sqlite3. And the nice thing about this documentation is it has plenty of good examples: http://docs.python.org/library/sqlite3.html Good luck, RD -- http://mail.python.org/mailman/listinfo/python-list
Re: Install Python MySQL db module?
On May 13, 7:29 pm, Con <[EMAIL PROTECTED]> wrote: > Hi, how does properly install the Python MySQL db module for Mac OS > X? I was only able to locate the Win32 modules. > > Thanks in advance, > > -Conrad I tried this a couple of weeks ago using macports and had problems. See, for example: http://www.davidcramer.net/code/57/mysqldb-on-leopard.html I read about several people who went in and edited the source files to work out the bugs on Leopard, but instead I just changed my Python code to use SQLite for the moment until someone fixes MySQLdb. There's a nice Perl script that will convert your MySQL db to an SQLite db if you're interested. RD -- http://mail.python.org/mailman/listinfo/python-list
Re: Command line input
On Mar 31, 2:39 pm, [EMAIL PROTECTED] wrote:
> How do I receive input from the command line in Python?
As long as we are all guessing, do you perhaps mean raw_input?
my_name = raw_input("What is your name? ")
What is your name? Rick
>>> my_name
'Rick'
--
http://mail.python.org/mailman/listinfo/python-list
Re: Recursively Backup Directories
On Apr 5, 6:56 pm, [EMAIL PROTECTED] wrote: > What I would like to do > is recursively backup the specified directories . . . > but be able to specify exclusion directories (which copytree does > not appear to allow you to do). My initial thoughts were I'll > probably have to use os.path.walk for the backup source directory, but > I'm not sure how to go from there. Thanks in advance. There's a nice Python Cookbook recipe. http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/191017 I think the one in the book is newer and better http://tinyurl.com/5vr4n6 And the Cookbook is my favorite way to learn Python. rd -- http://mail.python.org/mailman/listinfo/python-list
new python docs
Wow! I've been away in other pursuits. The new docs are gorgeous and searchable. http://docs.python.org/dev/index.html Thank you, python.org. -- http://mail.python.org/mailman/listinfo/python-list
Re: Verifying My Troublesome Linkage Claim between Python and Win7
No telling what Windows will do. :) I am a mere hobbyist programmer, but I think real programmers will tell you that it is a bad habit to use relative paths. Use absolute paths instead and remove all doubt. http://docs.python.org/library/os.path.html RD -- http://mail.python.org/mailman/listinfo/python-list
Re: Verifying My Troublesome Linkage Claim between Python and Win7
On Feb 23, 1:08 pm, Gib Bogle wrote: > It isn't useful to respond to a serious question with OS bigotry. Okay, I'll go with what Aahz said: > I've seen similar issues on Win7. > AFAIK, this has nothing to do with Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: help with Python installation
> Hi, > I have been trying to install python on my Win ME system Try this: http://tinyurl.com/w7wgp RD -- http://mail.python.org/mailman/listinfo/python-list
