Decorators and buffer flushing
Hi, all.
I apologize for what is perhaps a newb question. I'm in the process of
transitioning our testing framework from Perl to Python. While that alone
probably sets off some red flags, I'm afraid it's what I'm stuck with.
I'm modeling a test with five operations: build, execute, validate,
publish, and clean. The main loop might look something like this:
with Test(...) as t:
t.build()
t.execute()
t.validate()
t.publish()
At each run, I want to output a '.' to denote completion of that test
step. I've been able to do this, sort of, using the following decorator
(edited for brevity):
def report(f):
def new(self):
stat = f(self);
if stat is True:
sys.stdout.write ('.')
else:
...
sys.stdout.flush()
return new
(Each one of the test functions returns True or False depending on its
completion status.)
The problem is that rather than outputting a period after each step is
completed, it outputs all four of them at once. Instead of seeing
progress as it happens, I get it when it's finished, even though I'm
flushing the output buffer.
Any thoughts?
Best,
Ethan ([EMAIL PROTECTED])
http://uppertank.net/ethanm/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Decorators and buffer flushing
On Thu, 28 Feb 2008 14:48:55 -0500, <[EMAIL PROTECTED]> wrote:
> Can you reproduce the bug on the console? Or, try starting with a new
> line in the output. Or, try a input().
I can reproduce the issue in the console. I'm not convinced it's actually
a bug, unless for some reason the interpreter is preventing a buffer flush.
The new line in the output also doesn't work (and isn't desirable anyway),
and neither does adding input.
Could the problem be that I'm using a with statement? I have output
flushed in __enter__:
def __enter__(self):
"""Ensures that the test begins appropriately.
In particular, this method will enter the directory specified in the
instance."""
self.pardir = os.getcwd()
os.chdir(self.path)
sys.stdout.write ("%s" % (self.name.ljust(30),))
sys.stdout.flush()
return self
Is it possible that flushing is prohibited until __exit__ is called?
Best,
Ethan ([EMAIL PROTECTED])
http://uppertank.net/ethanm/
--
http://mail.python.org/mailman/listinfo/python-list
Re: Decorators and buffer flushing
On Thu, 28 Feb 2008 15:04:38 -0500, Ethan Metsger <[EMAIL PROTECTED]> wrote: > > I can reproduce the issue in the console. I'm not convinced it's > actually > a bug, unless for some reason the interpreter is preventing a buffer > flush. Quick question. Having eliminated some of the other variables in my code, I'm wondering if the following might cause a problem flushing the stdout buffer: proc = Popen (['ant'] + self.args, stdout=PIPE, stderr=PIPE) (out, err) = proc.communicate() Is it possible that the Popen object has somehow mangled things? For what it's worth, I'm also passing '-u' to the interpreter with no effect. Best, Ethan ([EMAIL PROTECTED]) http://uppertank.net/ethanm/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Decorators and buffer flushing
Hi, Gabriel. I missed this message initially; I apologize for not
responding sooner.
On Thu, 28 Feb 2008 18:53:28 -0500, Gabriel Genellina
<[EMAIL PROTECTED]> wrote:
>> I can reproduce the issue in the console. I'm not convinced it's
>> actually
>> a bug, unless for some reason the interpreter is preventing a buffer
>> flush.
>
> Try starting the interpreter with python -u xxx.py (-u = unbuffered) but
> I don't really think this might be the cause. Are you sure the code
> actually says sys.stdout.flush() and not sys.stdout.flush?
As I mentioned in another message, I haven't had any luck with adding '-u'
to the invocation of the interpreter. I am sure I'm not looking at the
callable object rather than calling flush().
>>sys.stdout.write ("%s" % (self.name.ljust(30),))
>
> Usually written as:
> sys.stdout.write("%-30s" % self.name)
Thanks for the tips!
[...]
>> Is it possible that flushing is prohibited until __exit__ is called?
>
> I don't think so... What's your platform/OS?
I'm running Ubuntu Feisty with Python 2.5.1:
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2
> Try to post a minimal example showing the problem - copy & paste it,
> don't retype. Maybe in the process of reducing your code to find the
> minimal example, you find the problem yourself.
The minimal example is a little long, but I think in the end I figured out
what was happening. The initial build step took the longest amount of
time to complete, but the other steps were nearly instantaneous.
So the post-build '.' would appear, and the very fast run of the other
steps made it appear as though everything was being buffered when it
wasn't. Adding in a short delay (time.sleep(0.05) seemed to work) helped
demonstrate that the data actually were unbuffered.
> Are you aware of the existing framework for unit tests? the unittest
> module?
Yes. I haven't investigated its uses in this context due to the
constraints of the legacy system and general inertia. I'm trying to
duplicate functionality while addressing certain annoyances with the
previous system.
Thanks again for your help!
Best,
Ethan
--
Ethan Metsger
http://uppertank.net/ethanm/
--
http://mail.python.org/mailman/listinfo/python-list
