Reducing cache/buffer for faster display
I have these 2 scripts that are very heavy on the file i/o, consume a very reasonable amount of cpu and output their counters at a - very - relaxed pace to the console. The output is very simply done using something like: print "files:", nFiles, "\r", Yet alltough there is no real reason for it, even a pace of a print every 10-30 secs will be cached, only to actually show an output update every 1-2 min or so. When I run the scripts with "python -u myscript.py", the output is complete, very speedy and without any kind of impact on the actual work being one. Can this option be called from within the script? Or is there another option to make the display "a bit" speedier ? Runnning Python 2.7.3, but it seems to me I've allready had this problem a long, long time ago with other releases. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: test
On 2012-09-27, Devin Jeanpierre wrote: > On Thu, Sep 27, 2012 at 5:28 PM, ForeverYoung wrote: >> Please ignore this post. >> I am testing to see if I can post successfully. > > Is there a reason you can't wait until you have something to say / ask > to see if it works? You're spamming a large number of inboxes with > nothing. Inboxes? What is this, usenet or email ? -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Reducing cache/buffer for faster display
On 2012-09-27, Chris Angelico wrote: > On Fri, Sep 28, 2012 at 7:57 AM, Rikishi42 wrote: >> I have these 2 scripts that are very heavy on the file i/o, consume a very >> reasonable amount of cpu and output their counters at a - very - relaxed >> pace to the console. The output is very simply done using something like: >> >>print "files:", nFiles, "\r", >> >> >> Yet alltough there is no real reason for it, even a pace of a print every >> 10-30 secs will be cached, only to actually show an output update every 1-2 >> min or so. > > Yup! Just add a call to sys.stdout.flush() after each print. Update: tried it, ran it, I love it. Thanks ! -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Reducing cache/buffer for faster display
On 2012-09-27, Chris Angelico wrote: > On Fri, Sep 28, 2012 at 8:25 AM, John Gordon wrote: >> Isn't terminal output line-buffered? I don't understand why there would >> be an output delay. (Unless the "\r" is messing things up...) > > This is a classic progress-indication case, which does indeed mess up > line-buffering. The carriage return (and no line feed, done in the > Python 2 style of a trailing comma) puts the cursor back to the > beginning of the line, ready to overwrite, and ripe for one of those > old favorite incomplete overwrite errors - if nFiles monotonically > increases, it's fine, but if it decreases, the display can get ugly. True, but that wasn't the problem here. The updates where. Thanks for the given answer, I'll try it. The scripts in question only increase numbers. But should that not be the case, solutions are simple enough. The numbers can be formatted to have a fixed size. In the case of random line contents (a list of filesnames, say) it's enough to create an output function that is aware of the length of the previously printed line, and add enough spaces to the current one to wipe exess content. Thanks again for the suggestion. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Reducing cache/buffer for faster display
On 2012-09-28, Dennis Lee Bieber wrote: > On Thu, 27 Sep 2012 22:25:39 + (UTC), John Gordon > declaimed the following in gmane.comp.python.general: > >> >> Isn't terminal output line-buffered? I don't understand why there would >> be an output delay. (Unless the "\r" is messing things up...) > > It's the trailing , The \r is being used to reset to the > beginning of the console line, but the comma "says" more output for > /this/ line will be coming... So no output until explicitly flushed, or > a new-line is issued. Well, the \r seems to be the problem, allright. But output was not completely blocked, just delayed a very long time. So perhaps flushing and a sending a newline aren't the only triggers for output. Perhaps there's a maximum delay or a maximum cumulated size, and the output is flushed when such a limit is reached. Anyway, that's mainly academic. I doubt there will be a correction to that behaviour. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Reducing cache/buffer for faster display
On 2012-09-28, Chris Angelico wrote: > On Fri, Sep 28, 2012 at 10:05 AM, Rikishi42 wrote: >> The scripts in question only increase numbers. But should that not be the >> case, solutions are simple enough. The numbers can be formatted to have a >> fixed size. In the case of random line contents (a list of filesnames, say) >> it's enough to create an output function that is aware of the length of the >> previously printed line, and add enough spaces to the current one to wipe >> exess content. > > Yep, that's a pretty effective way to do it. One simple method to it > is to format the whole string as a single whole, then left justify it > in a field of (say) 79 characters, and output that: > > msg = "Progress: %d%% (%d/%d)... %s" % (done*100/total, done, total, > current_file) > print msg.ljust(79)+"\r", > sys.stdout.flush() Mmm, I allmost went for that. It's elegant, simple and clear. But there's one drawback: I usually reduce the terminal's window to take up less desktop surface during those long runs. So fixing it to 79 chars won't do. And I'm not even tempted to go for a detection of the width of the terminal from within the script. The idea is after all to keep the scripts simple (syntax) and light (execution). Well, good night everyone. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Remove all directories using wildcard
On 2011-03-18, JSkinn3 wrote:
> I'm new to python and I am trying to figure out how to remove all sub
> directories from a parent directory using a wildcard. For example,
> remove all sub directory folders that contain the word "PEMA" from the
> parent directory "C:\Data".
>
> I've trying to use os.walk with glob, but I'm not sure if this is the
> right path to take.
I see you've got 2 suggestions allready.
Wichever you use, please note you need to begin the search from the bottom
of the tree. The call to os.walk should include a False value for the
direction of the walk:
os.walk("c:/data", False).
--
When in doubt, use brute force.
-- Ken Thompson
--
http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On 2011-05-18, Hans Georg Schaathun wrote: > Now Mac OS X has maintained the folder concept of older mac generations, > and Windows has cloned it. They do not want the user to understand > recursive data structures, and therefore, naturally, avoid the word. You imply they want to keep their users ignorant of these structures, as if to keep something valuable from them. Wouldn't it be more honest, more to the point and much simpler to state they don't NEED the user to understand recursive - or indeed any other - data structures? And that the user doesn't NEED to understand or know about them, just to use them? After all they are users. They use their system for fun, learning or work. Even a very competent or advanced use of a tool (computer, car, mobile phone, fridge, TV, radio, toilet) in no way implies an understanding of it's inner workings. Nor the need, nor the desire. PS: Isn't this thread much ado about nothing? :-) It starts with the misconception (or should I say confusion?) between performing a recursive job and using a recursive tool to do it. And then it blazes off in these huge discusions about semantics to define a definition of an abstraction of a alleady theoretical problem. Glorious, just frelling glorious.:-) We have an expression for that. But I'll avoid using it, since it has the word 'masturbation' in it... And PPS: the P(P)S's don't specifically refer to your posting. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On 2011-05-20, Hans Georg Schaathun wrote: >: It starts with the misconception (or should I say confusion?) between >: performing a recursive job and using a recursive tool to do it. And then it >: blazes off in these huge discusions about semantics to define a definition >: of an abstraction of a alleady theoretical problem. > > And explaining the source of the misconception and the varying use > would be irrelevant? It usually is, yes. And boring. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On 2011-05-20, Steven D'Aprano wrote: > On Thu, 19 May 2011 22:13:14 -0700, rusi wrote: > >> [I agree with you Xah that recursion is a technical word that should not >> be foisted onto lay users.] > > I think that is a patronizing remark that under-estimates the > intelligence of lay people and over-estimates the difficulty of > understanding recursion. Why would you presume this to be related to intelligence? The point was not about being *able* to understand, but about *needing* to understand in order to use. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On 2011-05-24, Steven D'Aprano wrote: >>> I think that is a patronizing remark that under-estimates the >>> intelligence of lay people and over-estimates the difficulty of >>> understanding recursion. >> >> Why would you presume this to be related to intelligence? The point was >> not about being *able* to understand, but about *needing* to understand >> in order to use. > > Maybe they don't "need" to understand recursion. So what? I think you should read the earlier posts again, this is drifting so far from what I intended. What I mean is: I'm certain that over the years I've had more than one person come to me and ask what 'Do you wish to delete this directory recursively?' meant. BAut never have I been asked to explain what 'Do you wish to delete this directory and it's subdirs/with all it's contents?' meant. Never. > Recursion is a perfectly good English word, no more technical than > "accelerate" or "incinerate" or "dissolve" or "combustion". Do people > need to know the word "combustion" when they could say "burn" instead? It wasn't about the word, but about the nature of the function. Besides, if the chance exists of a confusion between a recursive job and the fact the job is done using a recursive function... I would try staying away from the expression. Why not use 'delete a directory'. It's obvious the content gets binned, too. Do you know many people who incinerate leaves and branches in their garden? I burn them. > Do they need to know the words "microwave oven" when they could be saying > "invisible rays cooking thing"? The word oven has existed for ages, microwave is just a name for the type of oven. Not even a description, just a name. > I wonder whether physicists insist that cars should have a "go faster > pedal" because ordinary people don't need to understand Newton's Laws of > Motion in order to drive cars? Gas pedal. Pedal was allraedy known when the car was invented. The simple addition of gas solved that need. Oh, and it's break pedal, not descellarator. (sp?) > Who are you to say that people shouldn't be exposed to words you deem > that they don't need to know? I'm one of the 'people'. You say exposed to, I say bothered/bored with. I have nothing against the use of a proper, precise term. And that word can be a complex one with many, many sylables (seems to add value, somehow). But I'm not an academic, so I don't admire the pedantic use of terms that need to be explained to 'lay' people. Especially if there is a widespread, usually shorter and much simpler one for it. A pointless effort if pointless, even when comming from a physicist. :-) -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On 2011-05-25, Steven D'Aprano wrote: > I know many people who have no idea what a directory is, let alone a > subdirectory, unless it's the phone directory. They're non-computer > users. Once they start using computers, they quickly work out what the > word means in context, or they ask and get told, and then they've learned > a new word and never need ask again. This is a good thing. > > The idiom of "recursively delete" is no different. Of course some people > will have to learn a new term in order to make sense of it. So what? OK, but the addition of recursive, is not really usefull, is it? Deleting the directory says it, doesn't it? >> Do you know many people who incinerate leaves and branches in their >> garden? I burn them. > > I know many people who incinerate leaves in an incinerator. Or at least > they used to, until the government here banned it. It might only have > been a 44 gallon drum with holes punched in the side, but they still > called it an incinerator. > > I learned that word from my father, who left school at 14 to work in a > shoe shop. He isn't especially educated, doesn't read much beyond the > daily tabloid, and thinks Benny Hill is the height of wit. But he's not > an idiot and even at 72 is capable of learning new words. Is it that widespread? I figured most people woul speak of burning. OK, my bad if it is. >>> Do they need to know the words "microwave oven" when they could be >>> saying "invisible rays cooking thing"? >> >> The word oven has existed for ages, microwave is just a name for the >> type of oven. Not even a description, just a name. > > Why do you think they're called "microwave ovens" instead of "fizzbaz > ovens"? Could it possibly have something to do with the fact that they > cook with microwaves? > > So not actually "just a name" at all. It's a jargon description of the > implementation of the oven. True, but I meant that they just use it as a name. I don't think many people would actually try to find out what a microwave is. >>> I wonder whether physicists insist that cars should have a "go faster >>> pedal" because ordinary people don't need to understand Newton's Laws >>> of Motion in order to drive cars? >> >> Gas pedal. Pedal was allraedy known when the car was invented. The >> simple addition of gas solved that need. > > What's a gas pedal? Is that some strange American term for what most of > the English-speaking world knows as the accelerator? *wink* Oh, come one. I'm sure I've heard that often enough NOT to have imagined it. >> Oh, and it's break pedal, not descellarator. (sp?) > That would be brake, and decelerator. Sorry. But I actually have a excuse for those. (see below) ;-) >> I'm one of the 'people'. You say exposed to, I say bothered/bored with. > > You can't force people to learn new words, although you would be > surprised how even the most disinterested, lazy speaker manages to pick > up vocabulary without even being aware of it. I know, I'm one. > But nor do you have to pander to the slackers. They can learn the word, > or not, I don't care. If I'm writing for an audience of children, or > English as a second language, or the otherwise linguistically challenged, Third, actually. But I do try. ;-) > I'll simplify my vocabulary appropriately. For everyone else, I'll use an > ordinary adult vocabulary, and that includes the word "recursion" or > "recursive". It's hardly technical jargon -- I've found a discussion of > gangsta rap that uses it. Even children understand the concept of > recursion (self-reference). People put it in comedies like Blazing > Saddles and Space Balls! How difficult is it to put a name to the concept? > > >> I have nothing against the use of a proper, precise term. And that word >> can be a complex one with many, many sylables (seems to add value, >> somehow). >> >> But I'm not an academic, so I don't admire the pedantic use of terms >> that need to be explained to 'lay' people. > > Pedantic... that's another one of those academic words that need to be > explained to lay people, isn't it? As is academic itself, and in fact > "lay people". Who uses "lay people" in conversation? That (lay people) was atually a quote, from someone who actually used it in this thread. >> widespread, usually shorter and much simpler one for it. A pointless >> effort if pointless, even when comming from a physicist. :-) > > I think you *grossly* underestimate how many words people know, > particularly if you include so-called "passive vocabulary" (words people > can understand in context, but not define precisely). See, for example: > > http://www.worldwidewords.org/articles/howmany.htm Got a point, there. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On 2011-05-28, Chris Angelico wrote: > I think it's geographic. This list covers a lot of geography; I'm in > Australia, there are quite a few Brits, and probably the bulk of posts > come from either the US or Europe. (And yes, I did deliberately fold > all of Europe down to one entity, and I did also deliberately leave > Great Britain out of that entity.) I allways found that odd. Especially if you're talking geography, not politics. I can understand they want to be seen as independant, even they are in it enough to allways opose anything someone else suggests. :-) To me, saying the UK isn't part of Europe, is like saying Japan isn't part of Asia. Oh by the way, I'm Belgian. > Most things work out that way. A thing gets a name based either on its > implementation or on the brand name of the first/most popular one. If > the only microwave oven ever produced had been made by Foobar Corp, > and that company were not known for anything else, then quite possibly > everyone would call them "foobar ovens". Yeah, when I was a kid a photo camera was called a Kodak. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: English Idiom in Unix: Directory Recursively
On 2011-05-28, Chris Angelico wrote: > Chris Angelico > yes, bit of a Bible geek as well as a programming geek So you don't believe in genetic algorithms, then ? (ducking for cover) -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Need GUI pop-up to edit a (unicode ?) string
I'm in need for a graphical pop-up that will display a (unicode ?) string in a field, allow the user to change it and return the modified string. Maybe also keep the original one displayed above it. Something like this: +-+ | Please confirm or edit the following string | | | | Original_example_string | | | | +---+ | | | Original_about_to_be_changed | | | +---+ | | | | OK | | | +-+ I've never used any kind of graphical interface programing before, so I don't have a clue where to start. This would, however, be the *only* GUI part in the app at this point. >From what I can see the solution lays with PyQT, but the docs I find are courses that aim to teach the whole GUI tool. I only need a little pop-up to alow a user to edit part of a filename, for instance. I'm using Python 2.6.x on various Linux platforms (mainly openSUSE and Mint) and on Windows. Windows support is not important, in this case. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Need GUI pop-up to edit a (unicode ?) string
On 2011-01-22, Corey Richardson wrote: > On 01/22/2011 03:22 PM, Rikishi42 wrote: >> >> I'm in need for a graphical pop-up that will display a (unicode ?) string in >> a field, allow the user to change it and return the modified string. >> > If that is all you need, I suggest Tkinter. Nice and easy, comes built > into Python. Looks like you need two labels, an entry, and a button. > When I was learning Tkinter I used http://effbot.org/tkinterbook/. I had to add Tkinter, which was not isntalled on my machine. But it looks easy enough, I'll definitively look into it. Thanks ! -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Need GUI pop-up to edit a (unicode ?) string
On 2011-01-22, geremy condra wrote:
> If windows doesn't matter to you, just use Zenity. Here's a python
> function wrapping zenity that does what you want:
>
> import commands
>
> def confirm_or_edit(s):
> zenity = 'zenity'
> mode = '--entry'
> text = "--text='Please confirm or edit the following string:'"
> title = "--title='confirm or edit'"
> entry = "--entry-text='%s'" % s
> cmd = ' '.join([zenity, mode, text, title, entry])
> status, output = commands.getstatusoutput(cmd)
> if status: raise Exception("Couldn't run zenity")
> return output
>
> There's also a full-blown API for zenity, but this should do what you want.
Very, very nice. Thanks !
I'm amazed at how many GUI's are available.
No wonder I couldn't find "the" interface, there are too many. :-)
--
When in doubt, use brute force.
-- Ken Thompson
--
http://mail.python.org/mailman/listinfo/python-list
Re: Easy function, please help.
On 2011-02-09, Michael Hrivnak wrote: > Your function only works if n is an integer. Example: > num_digits(234) > 3 num_digits(23.4) > 325 > > When doing integer division, python will throw away the remainder and > return an int. Using your example of n==44, 44/10 == 4 and 4/10 == 0 > > Before each iteration of the while loop, the given expression (in this > case just n) is evaluated as a boolean. Your function would act the > same if it looked like this: > > def num_digits(n): > count = 0 > while bool(n): > count = count + 1 > n = n / 10 > return count > > 0 of course evaluates to False as a boolean, which is why the while loop > stops. > > Just for kicks, this function would work about as well: > > def num_digits(n): >return len(str(n)) What about this one: import math def num_digits(n): return int(math.ceil(math.log(n,10))) -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
Re: Easy function, please help.
On 2011-02-09, rantingrick wrote: > On Feb 9, 1:08�am, Paul Rudin wrote: >> Nanderson writes: >> > loop would be infinite. I get what is happening in the function, and I >> > understand why this would work, but for some reason it's confusing me >> > as to how it is exiting the loop after a certain number of times. Help >> > is appreciated, thanks. >> >> It works because 0 tests false and because integer division yields >> integers... eventually you'll get something like 1/10 giving 0. > > It works because of a design flaw in the language! Specifically i am > referring to treating the integer 0 as False and any other integers as > True. There is also the empty vs non empty containers that work this > way besides numeric types. I would have defined the flaw to be use of '/' for the integer division. Using 0 as false and any other value as true is hardly unique to python. Lots of languages have been doing this long before Python even existed. -- When in doubt, use brute force. -- Ken Thompson -- http://mail.python.org/mailman/listinfo/python-list
