Re: python response slow when running external DLL
jf...@ms4.hinet.net wrote:
> Peter Otten at 2015/11/27 UTC+8 8:20:54PM wrote：
>
>> Quick-fix example:
>> def download():
>> var.set("Starting download...")
>> root.update_idletasks()
>> time.sleep(3)
>> var.set("... done")
>
> Thanks, Peter, The update_idletasks() works. In my trivial program it's
> easy to apply for there are only two places call the DLL function.
>
>> A cleaner solution can indeed involve threads; you might adapt the
>> approach from (Python 2
>> code).
>
> Using thread is obviously more logical. I think my mistake was the "while
> busy: pass" loop which makes no sense because it blocks the main thread,
> just as the time.sleep() does. That's why in your link (and Laura's too)
> the widget.after() scheduling was used for this purpose.
No, the point of both recipes is that tkinter operations are only ever
invoked from the main thread. The main thread has polling code that
repeatedly looks if there are results from the helper thread. As far I
understand the polling method has the structure
f():
# did we get something back from the other thread?
# a queue is used to avoid race conditions
# if yes react.
# var_status.set() goes here
# reschedule f to run again in a few millisecs;
# that's what after() does
> From what I had learned here, the other way I can do is making the codes
> modified as follows. It will get ride of the "result" and "busy" global
> variables, but it also makes the codes looks a little ugly. I think I will
> take the update_idletasks() way in this porting for it seems more simpler,
> and can be used on thread or non-thread calling. Thank you again.
> .
> .
> #do the rest
> var_status.set('Download...')
> _thread.start_new_thread(td_download, ()) #must use threading
>
> def td_download():
> result = mydll.SayHello()
> if result:
> var_status.set("Download Fail at %s" % hex(result))
> showerror('Romter', 'Download Fail')
> else:
> var_status.set('Download OK')
> showinfo('Romter', 'Download OK')
As td_download() runs in the other thread the var_status.set() methods are
problematic.
Another complication that inevitably comes with concurrency: what if the
user triggers another download while one download is already running? If you
don't keep track of all downloads the message will already switch to
"Download OK" while one download is still running.
--
https://mail.python.org/mailman/listinfo/python-list
askopenfilename() (was: Re: non-blocking getkey?)
Ulli Horlacher wrote: > eryksun wrote: > > On Thu, Nov 19, 2015 at 10:31 AM, Michael Torrie wrote: > > > One windows it might be possible to use the win32 api to enumerate the > > > windows, find your console window and switch to it. > > > > You can call GetConsoleWindow [1] and then SetForegroundWindow [2]. > (...) > > great, this works! Thanks! One of my Windows test users reports, that the file dialog window of askopenfilename() starts behind the console window and has no focus. On Linux (XFCE) I do not have this problem. I start it with: Tk().withdraw() file = askopenfilename(title='select a file',initialdir=HOME) set_window_focus() # give focus back to console window Can one force askopenfilename() to start in foreground with focus? -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ -- https://mail.python.org/mailman/listinfo/python-list
Re: python response slow when running external DLL
In a message of Sat, 28 Nov 2015 11:13:38 +0100, Peter Otten writes: >jf...@ms4.hinet.net wrote: >> Using thread is obviously more logical. I think my mistake was the "while >> busy: pass" loop which makes no sense because it blocks the main thread, >> just as the time.sleep() does. That's why in your link (and Laura's too) >> the widget.after() scheduling was used for this purpose. I never saw the reply that Peter is replying to. The threading module constructs a higher level interface on top of the low level thread module. Thus it is the preferred way to go for standard Python code -- and even Fredrik's recipe contains the line: import thread # should use the threading module instead! Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: Looking for ideas to improve library API
Documentation is all you can do. -- Devin On Thu, Nov 26, 2015 at 5:35 AM, Chris Lalancette wrote: > On Thu, Nov 26, 2015 at 7:46 AM, Devin Jeanpierre > wrote: >> Why not take ownership of the file object, instead of requiring users >> to manage lifetimes? > > Yeah, I've kind of been coming to this conclusion. So my question > then becomes: how do I "take ownership" of it? I already keep a > reference to it, but how would I signal to the API user that they > should no longer use that file object (other than documentation)? > > Thanks, > Chris -- https://mail.python.org/mailman/listinfo/python-list
Re: askopenfilename()
Am 28.11.15 um 11:29 schrieb Ulli Horlacher:
One of my Windows test users reports, that the file dialog window of
askopenfilename() starts behind the console window and has no focus.
On Linux (XFCE) I do not have this problem.
I start it with:
Tk().withdraw()
file = askopenfilename(title='select a file',initialdir=HOME)
set_window_focus() # give focus back to console window
Can one force askopenfilename() to start in foreground with focus?
I can't test it right now, but I think it /should/ go into the
foreground by itself. For a toplevel window, which you create manually,
there is a lift() method which asks the OS to move the window to the
top. But on Windows, this file dialog is a native call and cannot be
influenced that much.
I see two thingd:
1) Tk().withdraw()
- this seems odd to me, because you don't keep a reference to the Tk
object around. Better do
root=Tk()
roo.withdraw()
2) Maybe it helps if you inject an update() after the withdraw(), maybe not
root.update()
3) I can confirm, that also on OSX the file dialog does not raise above
the console window, though it is not as bad because the dialog window is
muhc bigger then the console window.
I think that you are curing a lot of symptoms with the focus setting to
the console etc. Many problems would simply go away if you wrote the
whole thing as a GUI program. If I understand correctly, what you want -
a program to select files and folders to upload to your server - then
this would not be that much more work than the CLI with input() which
you are writing, and definitely less work to get it correct than the
plastering of the symptoms.
For example, a very simple approach would use a listbox with a + and a -
button. upon hitting - (or the delete key), you delete the selected
entries. Upon hitting +, you pop up the file selection dialog. upon
hitting a Go button, you send the files to the server. A minimalistic
version of it is included below. That is less then 50 lines of code,
including comments, without all the focus problems, providing a standard
desktop GUI metaphor. I haven't seen your command line code, but I doubt
that it is significantly simpler.
Of course, the code below can still use a great deal of polishing, like
scrollbars for the listbox, allowing multiple selection both for the
file dialog and the listbox, nice icons for the buttons, trapping the
close button on the main window with an "are you sure?"-type question,
maybe wrapping it up in a class, a progress bar during the upload and a
way to interrupt it... which is left as an exercise to the reader.
Christian
=
import Tkinter as tk, tkFileDialog as fd, ttk
from Tkinter import N,S,W,E
# create one ttk::frame to fill the main window
root=tk.Tk()
main=ttk.Frame(master=root)
# tell the pack geometry manager to completely
# fill the toplevel with this frame
main.pack(fill=tk.BOTH, expand=1)
# now create a listbox and a button frame
lb=tk.Listbox(master=main)
bf=ttk.Frame(master=main)
# use the grid manager to stack them, whereby
# the listbox should expand
lb.grid(row=0, column=0, sticky=(N,S,E,W))
bf.grid(row=1, column=0, sticky=(N,S,E,W))
main.rowconfigure(0, weight=1)
main.columnconfigure(0, weight=1)
def addfile():
filename=fd.askopenfilename()
if filename:
lb.insert(tk.END, filename)
def remove(*args):
sel=lb.curselection()
if sel:
lb.delete(sel)
def submit():
print("Submitting files:")
for filename in lb.get(0,tk.END):
print("Sending %s"%filename)
# create the three buttons
btnplus=ttk.Button(master=bf, text="+", command=addfile)
btnminus=ttk.Button(master=bf, text="-", command=remove)
btngo=ttk.Button(master=bf, text="Submit", command=submit)
btnplus.pack(side=tk.LEFT)
btnminus.pack(side=tk.LEFT)
btngo.pack(side=tk.LEFT)
# bind also the delete and Backspace keys
lb.bind('', remove)
lb.bind('', remove)
root.mainloop()
===
--
https://mail.python.org/mailman/listinfo/python-list
Re: askopenfilename()
Christian Gollwitzer wrote: > Am 28.11.15 um 11:29 schrieb Ulli Horlacher: > > One of my Windows test users reports, that the file dialog window of > > askopenfilename() starts behind the console window and has no focus. > > On Linux (XFCE) I do not have this problem. > > > > I start it with: > > > >Tk().withdraw() > >file = askopenfilename(title='select a file',initialdir=HOME) > >set_window_focus() # give focus back to console window > > > > Can one force askopenfilename() to start in foreground with focus? > > I can't test it right now, but I think it /should/ go into the > foreground by itself. This is what I think, too :-) But my test user reports me, it is not so. > I see two thingd: > > 1) Tk().withdraw() > > - this seems odd to me, because you don't keep a reference to the Tk > object around. I thought, I need it for Tk initialization. But true, it is superfluous > I think that you are curing a lot of symptoms with the focus setting to > the console etc. This is done after the file selection window is closed. This works. > Many problems would simply go away if you wrote the whole thing as a GUI > program. Too much hassle. The predecessor was a Perl/Tk program and I have had to invest 90% of the programming work into the GUI handling. No fun at all. Now, with fexit in Python, I skipped most of these problems. The only GUI part is the file selection. > If I understand correctly, what you want - a program to select files and > folders to upload to your server This is only one of the tasks. The main menu looks: [s] send a file or directory [g] get a file [c] change login data (user, server, auth-ID) [l] login with webbrowser [u] update fexit [h] help [q] quit (with more features to come in the future) And the CLI: framstag@juhu:~: ./fexit.py -h usage: fexit [-C "comment"] [-a container] file(s) recipient[,...] example: fexit flupp.avi frams...@rus.uni-stuttgart.de example: fexit -C "more data" -a labdata *.png x...@flupp.org,x...@flupp.org usage: fexit FEX-download-URL example: fexit http://fex.rus.uni-stuttgart.de/fop/jHn34yp7/flupp.avi -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 virtualenvs
On 2015-11-28, D.M. Procida wrote: > I have a new installation of Debian Jessie, with Python 2.7 and 3.4 > installed. > > I want to use Python 3.4 by default for most things, so I want > virtualenv to create Python 3.4 virtualenvs unless I ask it to > otherwise. > > It turns out that this seems to be inordinately complex. sudo apt-get remove python-virtualenv sudo apt-get install python3-virtualenv -- https://mail.python.org/mailman/listinfo/python-list
Re: askopenfilename()
Am 28.11.15 um 13:48 schrieb Ulli Horlacher: Christian Gollwitzer wrote: Many problems would simply go away if you wrote the whole thing as a GUI program. Too much hassle. The predecessor was a Perl/Tk program and I have had to invest 90% of the programming work into the GUI handling. No fun at all. As I see it, the program consists only of user interface - or is there any "algorithm" working behinds the scenes? Maybe you could pass the task on to somebody who enjoys GUI programming? Now, with fexit in Python, I skipped most of these problems. The only GUI part is the file selection. If I understand correctly, what you want - a program to select files and folders to upload to your server This is only one of the tasks. The main menu looks: [s] send a file or directory [g] get a file [c] change login data (user, server, auth-ID) [l] login with webbrowser [u] update fexit [h] help [q] quit All of this is easily integrated into a GUI like the one I posted (have you tried it?), either as a button or as a menu entry. IMO the most common GUI pattern for this kind of thing is a side-by-side view of the directories on the server and on the client, and a button (or drag'n'drop) to move files between both views. I understand this is not as easy as the script posted by me - nonetheless quite doable. For an experienced GUI script writer it'll take a weekend to get the basics running. (with more features to come in the future) And the CLI: framstag@juhu:~: ./fexit.py -h usage: fexit [-C "comment"] [-a container] file(s) recipient[,...] example: fexit flupp.avi frams...@rus.uni-stuttgart.de example: fexit -C "more data" -a labdata *.png x...@flupp.org,x...@flupp.org usage: fexit FEX-download-URL example: fexit http://fex.rus.uni-stuttgart.de/fop/jHn34yp7/flupp.avi This part should probably stay as it is. For a command line tool, an scp-like interface seems well-fitting. But for guided user input, an interface which prompts the user for input has never been a good solution. You have to work very hard to make that convenient. Have a look at lftp, for instance. In the end, real GUI programming will be easier (and more accessible) A (still) alternative solution would be an interface to the OS to make it a remote mounted folder (works for WebDAV on any modern OS, for instance) or a daemon, which watches and synchronizes a directory (this is how Dropbox works). This way it feels much more integrated to the user - they can use whatever file manager they like to do the transfer, or even "save" from any application (like Word, Firefox, ...) into the remote folder. Christian -- https://mail.python.org/mailman/listinfo/python-list
An Educational Software Platform written in Python
So after reading YCombinator's new RFS, I have decided I want to build an educational software platform written in Python (mostly, if not entirely). Here is the description from YCombinator's RFS : If we can fix education, we can eventually do everything else on this list. The first attempts to use technology to fix education have focused on using the Internet to distribute traditional content to a wider audience. This is good, but the Internet is a fundamentally different medium and capable of much more. Solutions that combine the mass scale of technology with one-on-one in-person interaction are particularly interesting to us. This may not require a "breakthrough" technology in the classical sense, but at a minimum it will require very new ways of doing things. What resources would I need to obtain, learn and utilise on order to create this platform? Can I create something like this entirely in Python, or would I need to learn other technologies? Anyone can give some directions, thanks alot ! Gengyang -- https://mail.python.org/mailman/listinfo/python-list
Re: askopenfilename()
Maybe Wei Li Jiang's hack will work for you? http://stackoverflow.com/questions/3375227/how-to-give-tkinter-file-dialog-focus But then see if it works under MacOS. I fear it will not. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: askopenfilename()
Ulli Horlacher wrote: > One of my Windows test users reports, that the file dialog window of > askopenfilename() starts behind the console window and has no focus. I have got a followup: this happens only with Windows XP, not with Windows 7. Therefore I will ignore this problem :-) -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Find relative url in mixed text/html
Hi Paul, On 28/11/15 13:11, Paul Rubin wrote: > Rob Hills writes: >> Note, in the beginning of this project, I looked at using "Beautiful >> Soup" but my reading and limited testing lead me to believe that it is >> designed for well-formed HTML/XML and therefore was unsuitable for the >> text/html soup I have. If that belief is incorrect, I'd be grateful for >> general tips about using Beautiful Soup in this scenario... > Beautiful Soup can deal with badly formed HTML pretty well, or at least > it could in earlier versions. It gives you several different parsing > options to choose from now. I think the default is lxml which is fast > but maybe more strict. Check what the others are and see if a loose > slow one is still there. It really is pretty slow so plan on a big > computation task if you're converting a large forum. I've had another look at Beautiful Soup and while it doesn't really help me much with urls (relative or absolute) embedded within text, it seems to do a good job of separating out links from the rest, so that could be useful in itself. WRT time, I'm converting about 65MB of data which currently takes 14 seconds (on a 3yo laptop with a SSD running Ubuntu), which I reckon is pretty amazing performance for Python3, especially given my relatively crude coding skills. It'll be interesting to see if using Beautiful Soup adds significantly to that. > phpBB gets a bad rap that's maybe well-deserved but I don't know what to > suggest instead. I did start to investigate Python-based alternatives; I've not heard much good said about php, but I probably move in the wrong circles. However, our hosting service doesn't support Python so I stopped hunting. Plus there is a significant group of forum members who hold very strong opinions about the functionality they want and it took a lot of work to get them to agree on something! All that said, I'd be interested to see specific (and hopefully unbiased) info about phpBB's failings... Cheers, -- Rob Hills Waikiki, Western Australia -- https://mail.python.org/mailman/listinfo/python-list
Re: An Educational Software Platform written in Python
Cai Gengyang : > Can I create something like this entirely in Python, Absolutely. It will only take ten to one hundred years for one person to create. Marko -- https://mail.python.org/mailman/listinfo/python-list
Re: askopenfilename()
Christian Gollwitzer wrote: > Am 28.11.15 um 13:48 schrieb Ulli Horlacher: > > Christian Gollwitzer wrote: > >> Many problems would simply go away if you wrote the whole thing as a GUI > >> program. > > > > Too much hassle. > > The predecessor was a Perl/Tk program and I have had to invest 90% of the > > programming work into the GUI handling. No fun at all. > > As I see it, the program consists only of user interface - or is there > any "algorithm" working behinds the scenes? There is a lot of "algorithms", about 50 kB of code: - resuming upload and download after link failures - sending multiple files as zip or tar containers - deleting files - configuring login - handling HTTP proxy - many other things... > Maybe you could pass the task on to somebody who enjoys GUI programming? No, I then have to maintain it. I do not like to support foreign code. I was in this situation already. The client was written in Java, which I do not know. I have had to throw it away some day, because I was not able to fix the bugs. Therefore I started to write the new client fexit in Python. Here we are :-) > > This is only one of the tasks. The main menu looks: > > > > [s] send a file or directory > > [g] get a file > > [c] change login data (user, server, auth-ID) > > [l] login with webbrowser > > [u] update fexit > > [h] help > > [q] quit > > All of this is easily integrated into a GUI like the one I posted (have > you tried it?) As I wrote: I have done this already with Perl/Tk and it was HASSLE. No fun at all. > IMO the most common GUI pattern for this kind of thing is a side-by-side > view of the directories on the server and on the client There is no directory view on server side. > For an experienced GUI script writer it'll take a weekend to get the > basics running. I am not even a beginner GUI script writer and I do not want to become one. GUIs are most superfluous and delay the work flow. > for guided user input, an interface which prompts the user for input has > never been a good solution. You have to work very hard to make that > convenient. This is easy, because they have no alternative :-) > Have a look at lftp, for instance. Yes. Horrible user interface and even more horrible to program :-} > A (still) alternative solution would be an interface to the OS to make > it a remote mounted folder There are no remote folders. > (works for WebDAV on any modern OS, for instance) or a daemon, which > watches and synchronizes a directory (this is how Dropbox works). A VERY bad idea, if you have to send TB files. > This way it feels much more integrated to the user It shall not look integrated. The user should think before using it. > they can use whatever file manager they like to do the transfer There are no file manager which supports the F*EX protocol. Therefore I am forced to write my own clients. I am a lazy guy: if there is already a program, I will happily use it. Only if there is none, I program it by myself. > or even "save" from any application (like Word, Firefox, ...) into the > remote folder. There are no remote folders. -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Find relative url in mixed text/html
In a message of Sun, 29 Nov 2015 00:25:07 +0800, Rob Hills writes: >All that said, I'd be interested to see specific (and hopefully >unbiased) info about phpBB's failings... People I know of who run different bb software say that the spammers really prefer phpBB. So keeping it spam free is about 4 times the work as for, for instance, IPB. Hackers seem to like it too -- possibly due to this: http://defensivedepth.com/2009/03/03/anatomy-of-a-hack-the-phpbbcom-attack/ make sure you aren't vulnerable. -- https://mail.python.org/mailman/listinfo/python-list
Re: Find relative url in mixed text/html
Hi Laura, On 29/11/15 01:04, Laura Creighton wrote: > In a message of Sun, 29 Nov 2015 00:25:07 +0800, Rob Hills writes: >> All that said, I'd be interested to see specific (and hopefully >> unbiased) info about phpBB's failings... > People I know of who run different bb software say that the spammers > really prefer phpBB. So keeping it spam free is about 4 times the > work as for, for instance, IPB. > > Hackers seem to like it too -- possibly due to this: > http://defensivedepth.com/2009/03/03/anatomy-of-a-hack-the-phpbbcom-attack/ > > make sure you aren't vulnerable. Thanks for the link and the advice. Personally, I'd rather go with something based on a language I am reasonably familiar with (eg Python or Java) however it seems the vast bulk of Forum software is based on PHP :-( Cheers, -- Rob Hills Waikiki, Western Australia -- https://mail.python.org/mailman/listinfo/python-list
Re: Find relative url in mixed text/html
Hi Grobu, On 28/11/15 15:07, Grobu wrote: > Is it safe to assume that all the relative (cross) links take one of > the following forms? : > > http://www.aeva.asn.au/forums/forum_posts.asp > www.aeva.asn.au/forums/forum_posts.asp > /forums/forum_posts.asp > /forum_posts.asp (are you really sure about this one?) > > If so, and if your goal boils down to converting all instances of old > style URLs to new style ones regardless of the context where they > appear, why would a regex fail to meet your needs? I'm actually not discounting anything and as I mentioned, I've already used some regex to extract the properly-formed URLs (those starting with http://). I was fortunately able to find some example regex that I could figure out enough to tweak for my purpose. Unfortunately, my small brain hurts whenever I try and understand what a piece of regex is doing and I don't like having bits in my code that hurt my brain. BTW, that's not meant to be an invitation to someone to produce some regex for me, if I can't find any other way of doing it, I'll try and create my own regex and come back here if I can't get that working. Cheers, -- Rob Hills Waikiki, Western Australia -- https://mail.python.org/mailman/listinfo/python-list
Re: Find relative url in mixed text/html
Rob Hills writes: > Personally, I'd rather go with something based on a language I am > reasonably familiar with (eg Python or Java) however it seems the vast > bulk of Forum software is based on PHP :-( It's certainly possible to write good software in PHP, so it's mostly a matter of the design and implementation quality. I was on a big PhpBB forum years ago and it got very slow as the database got large, and there were multiple incidents of database corruption. The board eventually switched to VBB which was a lot better. VBB is the best one I know of but it's not FOSS. I'm on another one right now which uses IPB (also not FOSS) and don't like it much (too clever for its own good). Another one is FluxBB which is nice and lightweight and FOSS, but it's a small forum and the software might not be up to handling a bigger one. Some people like Discourse. I don't like it much myself, but that's just me. There's certainly plenty of cheap hosting available these days (or raw VPS) that let you run Python or whatever else you want. But it seems to me that forum software is something of a ghetto. I do think there is some written in Python but I don't remember any specifics. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 virtualenvs
Jon Ribbens wrote: > On 2015-11-28, D.M. Procida wrote: > > I have a new installation of Debian Jessie, with Python 2.7 and 3.4 > > installed. > > > > I want to use Python 3.4 by default for most things, so I want > > virtualenv to create Python 3.4 virtualenvs unless I ask it to > > otherwise. > > > > It turns out that this seems to be inordinately complex. > > sudo apt-get remove python-virtualenv > sudo apt-get install python3-virtualenv Yup, I did try installing python3-virtualenv, but it didn't appear actually do do anything. It didn't provide me with a virtualenv command, that's for sure. And pages such as https://packages.debian.org/jessie/python3-virtualenv are not exactly informative. Is something else required? Daniele -- https://mail.python.org/mailman/listinfo/python-list
Re: An Educational Software Platform written in Python
On Sat, Nov 28, 2015 at 11:39 AM, Marko Rauhamaa wrote: > Cai Gengyang : > > > Can I create something like this entirely in Python, > > Absolutely. It will only take ten to one hundred years for one person to > create. > > > Marko > -- > https://mail.python.org/mailman/listinfo/python-list > Cai, Start small. write some 10 line programs. Then some 30 line programs. then write 100 more usefull programs. Then you will understand what it is you can do or can't. -- Joel Goldstick http://joelgoldstick.com/stats/birthdays -- https://mail.python.org/mailman/listinfo/python-list
Generate config file from template using Python search and replace.
I need to generate a config file based on an existing "template" file. I need
to replace a set of strings with other strings globally in the generated file.
Here is a snippet of the template file, where CONTENT_PATH and DAMPATH are two
"placeholders" or variables. There are several other such placeholders.
$include "_dispatcher_publish_filters.any"
/1000 { /type "allow" /glob "* /CONTENT_PATH/*.html*" }
/1001 { /type "allow" /glob "POST /DAMPATH/www/*.html *" }
The script's user will be asked to type in unique values when prompted for
DAMPATH or CONTENT_PATH.
Since I know the variables themselves are not going to change (because the
contents of the template file don't spontaneously change) should I be using
regex to search for them or is there a better way? I was planning on using
re.sub but I don't know whether that's the best way. Here's what my script
looks like today.
from sys import argv
import re
from os.path import exists
script, template_file = argv
print "Opening the template file..."
in_file = open(template_file)
lines = in_file.readlines()
print "What is the serial number of the site?",
_NNN = raw_input()
print "What is the brand, or product name?",
_BRAND = raw_input()
print "What is the content path?",
_CONTENT_PATH = raw_input()
out_file = open(_nnn + _brand + "_farm.any", 'w')
for line in lines:
re.sub('NNN', _NNN, line)
re.sub('BRAND, _BRAND', line)
re.sub('CONTENT_PATH', _CONTENT_PATH, line)
out_file.close()
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 virtualenvs
On 2015-11-28, D.M. Procida wrote: > Jon Ribbens wrote: > >> On 2015-11-28, D.M. Procida > wrote: >> > I have a new installation of Debian Jessie, with Python 2.7 and 3.4 >> > installed. >> > >> > I want to use Python 3.4 by default for most things, so I want >> > virtualenv to create Python 3.4 virtualenvs unless I ask it to >> > otherwise. >> > >> > It turns out that this seems to be inordinately complex. >> >> sudo apt-get remove python-virtualenv >> sudo apt-get install python3-virtualenv > > Yup, I did try installing python3-virtualenv, but it didn't appear > actually do do anything. It didn't provide me with a virtualenv command, > that's for sure. > > And pages such as https://packages.debian.org/jessie/python3-virtualenv > are not exactly informative. > > Is something else required? Hmm. Well basically the answer to your question is that you want virtualenv to be installed by Python 3, then it will default to using it. Debian's package management is mysterious and apparently bizarre and frankly in respect to Python, not very good. So perhaps the best thing to do is simply ignore it and install virtualenv yourself with: sudo apt-get install python3-pip sudo pip3 install virtualenv ... although you might want to install pip manually too. -- https://mail.python.org/mailman/listinfo/python-list
Re: Generate config file from template using Python search and replace.
I should mention the template file is small, just 98 lines long and the working config file will be the same size. -- https://mail.python.org/mailman/listinfo/python-list
Does Python allow variables to be passed into function for dynamic screen scraping?
I'm looking at this URL.
https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
If I hit F12 I can see tags such as these:
https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
r = requests.get(url)
soup=BeautifulSoup(r.content,"lxml")
#set up a function to parse the "soup" for each category of information and put
it in a DataFrame
def get_match_info(soup,tag,class_name):
info_array=[]
for info in soup.find_all('%s'%tag,attrs={'class':'%s'%class_name}):
return pd.DataFrame(info_array)
#for each category pass the above function the relevant information i.e. tag
names
tag1 = get_match_info(soup,"td","title")
tag2 = get_match_info(soup,"td","class")
#Concatenate the DataFrames to present a final table of all the above info
match_info = pd.concat([tag1,tag2],ignore_index=False,axis=1)
print match_info
I'd greatly appreciate any help with this.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Does Python allow variables to be passed into function for dynamic screen scraping?
In a message of Sat, 28 Nov 2015 14:03:10 -0800, ryguy7272 writes:
>I'm looking at this URL.
>https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
>
>If I hit F12 I can see tags such as these:
>And so on and so forth.
>
>I'm wondering if someone can share a script, or a function, that will allow me
>to pass in variables and download (or simply print) the results. I saw a
>sample online that I thought would work, and I made a few modifications but
>now I keep getting a message that says: ValueError: All objects passed were
>None
>
>Here's the script that I'm playing around with.
>
>import requests
>import pandas as pd
>from bs4 import BeautifulSoup
>
>#Get the relevant webpage set the data up for parsing
>url = "https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
>r = requests.get(url)
>soup=BeautifulSoup(r.content,"lxml")
>
>#set up a function to parse the "soup" for each category of information and
>put it in a DataFrame
>def get_match_info(soup,tag,class_name):
>info_array=[]
>for info in soup.find_all('%s'%tag,attrs={'class':'%s'%class_name}):
>return pd.DataFrame(info_array)
>
>#for each category pass the above function the relevant information i.e. tag
>names
>tag1 = get_match_info(soup,"td","title")
>tag2 = get_match_info(soup,"td","class")
>
>#Concatenate the DataFrames to present a final table of all the above info
>match_info = pd.concat([tag1,tag2],ignore_index=False,axis=1)
>
>print match_info
>
>I'd greatly appreciate any help with this.
Post your error traceback. If you are getting Value Errors about None,
then probably something you expect to return a match, isn't. But without
the actual error, we cannot help much.
Laura
--
https://mail.python.org/mailman/listinfo/python-list
Re: Does Python allow variables to be passed into function for dynamic screen scraping?
On Saturday, November 28, 2015 at 5:28:55 PM UTC-5, Laura Creighton wrote:
> In a message of Sat, 28 Nov 2015 14:03:10 -0800, ryguy7272 writes:
> >I'm looking at this URL.
> >https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
> >
> >If I hit F12 I can see tags such as these:
> > > >And so on and so forth.
> >
> >I'm wondering if someone can share a script, or a function, that will allow
> >me to pass in variables and download (or simply print) the results. I saw a
> >sample online that I thought would work, and I made a few modifications but
> >now I keep getting a message that says: ValueError: All objects passed were
> >None
> >
> >Here's the script that I'm playing around with.
> >
> >import requests
> >import pandas as pd
> >from bs4 import BeautifulSoup
> >
> >#Get the relevant webpage set the data up for parsing
> >url = "https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
> >r = requests.get(url)
> >soup=BeautifulSoup(r.content,"lxml")
> >
> >#set up a function to parse the "soup" for each category of information and
> >put it in a DataFrame
> >def get_match_info(soup,tag,class_name):
> >info_array=[]
> >for info in soup.find_all('%s'%tag,attrs={'class':'%s'%class_name}):
> >return pd.DataFrame(info_array)
> >
> >#for each category pass the above function the relevant information i.e. tag
> >names
> >tag1 = get_match_info(soup,"td","title")
> >tag2 = get_match_info(soup,"td","class")
> >
> >#Concatenate the DataFrames to present a final table of all the above info
> >match_info = pd.concat([tag1,tag2],ignore_index=False,axis=1)
> >
> >print match_info
> >
> >I'd greatly appreciate any help with this.
>
> Post your error traceback. If you are getting Value Errors about None,
> then probably something you expect to return a match, isn't. But without
> the actual error, we cannot help much.
>
> Laura
Ok. How do I post the error traceback? I'm using Spyder Python 2.7.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Does Python allow variables to be passed into function for dynamic screen scraping?
In a message of Sat, 28 Nov 2015 14:37:26 -0800, ryguy7272 writes:
>On Saturday, November 28, 2015 at 5:28:55 PM UTC-5, Laura Creighton wrote:
>> In a message of Sat, 28 Nov 2015 14:03:10 -0800, ryguy7272 writes:
>> >I'm looking at this URL.
>> >https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names
>> >
>> >If I hit F12 I can see tags such as these:
>> >> >> >And so on and so forth.
>> >
>> >I'm wondering if someone can share a script, or a function, that will allow
>> >me to pass in variables and download (or simply print) the results. I saw
>> >a sample online that I thought would work, and I made a few modifications
>> >but now I keep getting a message that says: ValueError: All objects passed
>> >were None
>> >
>> >Here's the script that I'm playing around with.
>> >
>> >import requests
>> >import pandas as pd
>> >from bs4 import BeautifulSoup
>> >
>> >#Get the relevant webpage set the data up for parsing
>> >url = "https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names";
>> >r = requests.get(url)
>> >soup=BeautifulSoup(r.content,"lxml")
>> >
>> >#set up a function to parse the "soup" for each category of information and
>> >put it in a DataFrame
>> >def get_match_info(soup,tag,class_name):
>> >info_array=[]
>> >for info in soup.find_all('%s'%tag,attrs={'class':'%s'%class_name}):
>> >return pd.DataFrame(info_array)
>> >
>> >#for each category pass the above function the relevant information i.e.
>> >tag names
>> >tag1 = get_match_info(soup,"td","title")
>> >tag2 = get_match_info(soup,"td","class")
>> >
>> >#Concatenate the DataFrames to present a final table of all the above info
>> >match_info = pd.concat([tag1,tag2],ignore_index=False,axis=1)
>> >
>> >print match_info
>> >
>> >I'd greatly appreciate any help with this.
>>
>> Post your error traceback. If you are getting Value Errors about None,
>> then probably something you expect to return a match, isn't. But without
>> the actual error, we cannot help much.
>>
>> Laura
>
>
>Ok. How do I post the error traceback? I'm using Spyder Python 2.7.
You cut and paste it out of wherever you are reading it, and paste it
into the email, along with your code, also cut and pasted from somewhere
(like an editor). That way we get the exact code that caused the exact
traceback you are getting.
Laura
--
https://mail.python.org/mailman/listinfo/python-list
Re: Generate config file from template using Python search and replace.
Mr Zaug wrote:
> I need to generate a config file based on an existing "template" file. I
> need to replace a set of strings with other strings globally in the
> generated file.
>
> Here is a snippet of the template file, where CONTENT_PATH and DAMPATH are
> two "placeholders" or variables. There are several other such
> placeholders.
>
> $include "_dispatcher_publish_filters.any"
> /1000 { /type "allow" /glob "* /CONTENT_PATH/*.html*" }
> /1001 { /type "allow" /glob "POST /DAMPATH/www/*.html *" }
>
> The script's user will be asked to type in unique values when prompted for
> DAMPATH or CONTENT_PATH.
>
> Since I know the variables themselves are not going to change (because the
> contents of the template file don't spontaneously change) should I be
> using regex to search for them or is there a better way? I was planning on
> using re.sub but I don't know whether that's the best way. Here's what my
> script looks like today.
>
> from sys import argv
> import re
> from os.path import exists
>
> script, template_file = argv
> print "Opening the template file..."
>
> in_file = open(template_file)
> lines = in_file.readlines()
>
> print "What is the serial number of the site?",
> _NNN = raw_input()
>
> print "What is the brand, or product name?",
> _BRAND = raw_input()
>
> print "What is the content path?",
> _CONTENT_PATH = raw_input()
>
> out_file = open(_nnn + _brand + "_farm.any", 'w')
>
> for line in lines:
>re.sub('NNN', _NNN, line)
>re.sub('BRAND, _BRAND', line)
>re.sub('CONTENT_PATH', _CONTENT_PATH, line)
>
> out_file.close()
There are many templating systems out there. Pick the one that suits you
best. A very basic one is str.format():
>>> "{foo} {bar}".format(foo=1, bar=2)
'1 2'
Here's what your script might become if you choose that one:
$ cat interactive_template.txt
$include "_dispatcher_publish_filters.any"
/1000 {{ /type "allow" /glob "* /{CONTENT_PATH}/*.html*" }}
/1001 {{ /type "allow" /glob "POST /{DAMPATH}/www/*.html *" }}
$ cat interactive_template.py
try:
format_map = str.format_map
except AttributeError: # python 2 compatibility
import string
def format_map(text, lookup):
return string.Formatter().vformat(text, (), lookup)
input = raw_input
class Lookup(dict):
def __missing__(self, key):
self[key] = value = input("{}: ".format(key))
return value
def main():
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("template_file")
parser.add_argument("generated_file", nargs="?")
args = parser.parse_args()
with open(args.template_file) as instream:
template_text = instream.read()
lookup = Lookup()
generated_text = format_map(template_text, lookup)
generated_file = args.generated_file
if generated_file is None:
generated_file = format_map("{nnn}{brand}_farm.any", lookup)
print("Writing {}".format(generated_file))
with open(generated_file, "w") as outstream:
outstream.write(generated_text)
if __name__ == "__main__":
main()
$ python interactive_template.py interactive_template.txt
CONTENT_PATH: foo
DAMPATH: bar
nnn: baz
brand: ham
Writing bazham_farm.any
$ cat bazham_farm.any
$include "_dispatcher_publish_filters.any"
/1000 { /type "allow" /glob "* /foo/*.html*" }
/1001 { /type "allow" /glob "POST /bar/www/*.html *" }
--
https://mail.python.org/mailman/listinfo/python-list
Re: Does Python allow variables to be passed into function for dynamic screen scraping?
On Sun, 29 Nov 2015 09:03 am, ryguy7272 wrote: > I'm looking at this URL. > https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names Don't screen-scrape Wikipedia. Just don't. They have an official API for downloading content, use it. There's even a Python library for downloading from Wikipedia and other Mediawiki sites: https://www.mediawiki.org/wiki/Manual:Pywikibot Wikimedia does a fantastic job, for free, and automated screen-scraping hurts their ability to provide that service. It is rude and anti-social. Please don't do it. -- Steven -- https://mail.python.org/mailman/listinfo/python-list
Re: python response slow when running external DLL
Peter Otten at 2015/11/28 UTC+8 6:14:09PM wrote：
> No, the point of both recipes is that tkinter operations are only ever
> invoked from the main thread. The main thread has polling code that
> repeatedly looks if there are results from the helper thread. As far I
> understand the polling method has the structure
>
> f():
># did we get something back from the other thread?
># a queue is used to avoid race conditions
>
># if yes react.
># var_status.set() goes here
>
># reschedule f to run again in a few millisecs;
># that's what after() does
Have no idea how the main thread poll on all those events (or it use a queue)?
All I know now is that the main thread(mainloop()?) can be easily blocked by
event handlers if the handler didn't run as a separate thread.
> > .
> > .
> > #do the rest
> > var_status.set('Download...')
> > _thread.start_new_thread(td_download, ()) #must use threading
> >
> > def td_download():
> > result = mydll.SayHello()
> > if result:
> > var_status.set("Download Fail at %s" % hex(result))
> > showerror('Romter', 'Download Fail')
> > else:
> > var_status.set('Download OK')
> > showinfo('Romter', 'Download OK')
>
> As td_download() runs in the other thread the var_status.set() methods are
> problematic.
No idea what kind of problem it will encounter. Can you explain?
> Another complication that inevitably comes with concurrency: what if the
> user triggers another download while one download is already running? If you
> don't keep track of all downloads the message will already switch to
> "Download OK" while one download is still running.
Hummm...this thought never comes to my mind. After take a quick test I found,
you are right, a second "download" was triggered immediately. That's a shock to
me. I suppose the same event shouldn't be triggered again, or at least not
triggered immediately, before its previous handler was completed. ...I will
take a check later on Borland C++ builder to see how it reacts!
Anyway to prevent this happens? if Python didn't take care it for us.
--
https://mail.python.org/mailman/listinfo/python-list
Re: python response slow when running external DLL
Laura Creighton at 2015/11/28 UTC+8 6:52:25PM wrote： > I never saw the reply that Peter is replying to. > The threading module constructs a higher level interface on top of the > low level thread module. Thus it is the preferred way to go for > standard Python code -- and even Fredrik's recipe contains the > line: > import thread # should use the threading module instead! > > Laura Hi! Laura, takes the porting of an old BCB GUI program as an exercise in my learning python, I just quickly grab the required tools (mainly the tkinter) in python to complete this "work" and get a feeling of how python performs on doing this. Most of my knowledge of python comes from Mark Lutz's book "Learning python" and "Programming python". I didn't dive into this language deeply yet. There are topics about "_thread" and "threading" modules in the book and I just pick the lower level one for this because the "threading" is based on the "_thread". Thanks for your note and link. --Jach -- https://mail.python.org/mailman/listinfo/python-list
Re: Does Python allow variables to be passed into function for dynamic screen scraping?
On Saturday, November 28, 2015 at 8:59:04 PM UTC-5, Steven D'Aprano wrote: > On Sun, 29 Nov 2015 09:03 am, ryguy7272 wrote: > > > I'm looking at this URL. > > https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names > > Don't screen-scrape Wikipedia. Just don't. They have an official API for > downloading content, use it. There's even a Python library for downloading > from Wikipedia and other Mediawiki sites: > > https://www.mediawiki.org/wiki/Manual:Pywikibot > > Wikimedia does a fantastic job, for free, and automated screen-scraping > hurts their ability to provide that service. It is rude and anti-social. > Please don't do it. > > > > -- > Steven Thanks Steven. Do you know of a good tutorial for learning about Wikipedia APIs? I'm not sure where to get started on this topic. I did some Google searches, but didn't come up with a lot of useful info...not much actually... -- https://mail.python.org/mailman/listinfo/python-list
