Re: [Python-Dev] Using logging in the stdlib and its unit tests
Gregory P. Smith krypto.org> writes:
> Hahaha. :) Well, I won't be suggesting to anyone at work that we throw away
> our entire bazillion line codebase just because all of it happily relies on
> logging.{debug,info,warn,error,exception} functions and all log messages go
> through a single root logger.
That's a perfectly valid way of using logging: if the convenience functions suit
your needs, there's no need to use anything else.
> I'd argue that anyone using a multi-logger hierarchy has already implemented
> overkill and that the default for anyone wanting to log something should be to
> simply call the above functions directly from the logging module.
Some people need the logger hierarchy and it's there for them, but no-one is
forced to use it.
>
> This simplistic easy usage somewhat echo's Glenn's comment on this thread
about logging seeming way to daunting as presented today. It needn't be.
>
Indeed, and the very first code sample in the logging documentation shows
exactly the simplistic easy usage you're talking about. I can't see why anyone
would be scared off by that example.
Regards,
Vinay Sajip
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/10/2010 12:06 AM, Vinay Sajip wrote: > This simplistic easy usage somewhat echo's Glenn's comment on this thread about logging seeming way to daunting as presented today. It needn't be. > Indeed, and the very first code sample in the logging documentation shows exactly the simplistic easy usage you're talking about. I can't see why anyone would be scared off by that example. It didn't scare me off. But it wasn't _quite_ comprehensive enough to keep me from being put off by the next bit of manual, (about 8%, as mentioned before), which had lots of discussion, but no more examples, so what is lacking up front is a few more examples of using the convenience methods as Greg apparently does, without configuration, but showing use in multiple modules. After it is shown to be simple to get started in a multi-module code base, then all the refinements can be explained and added, but by then people are already using the logger, and learning becomes incremental. Otherwise, people are scared off or put off by the need to learn lots of new terms (logger, filter, handler, formatter, and configuration) before they get any value out of it. The next thing after the simple example is a rotating log configuration. A handy feature, of course, but hardly the second thing that one needs to learn about logging. Maybe the 4th or 5th or later. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/9/2010 8:29 PM, Gregory P. Smith wrote:
Exactly. All I ever recommend people do is:
import logging
...
logging.warn('doing something a bit odd.')
...
for x in thing:
logging.debug('working on %r', x)
...
And be done with it. If they are controlling their __main__ they'll
probably want to call a common function to setup the log message
formatting and where it gets written to from there.
Greg -- can you flesh out that last line, to remove the need for my
parenthetical uncertainties below)?
So with your warn and debug calls, and the "common function to setup
..." (whatever this is, and however often and from where it needs to be
called in a multi-module environment) at the top of the logging manual,
it might be more approachable.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Vinay Sajip writes: > Indeed, and the very first code sample in the logging documentation > shows exactly the simplistic easy usage you're talking about. I > can't see why anyone would be scared off by that example. They're not scared by that example. What you need is a paragraph below it that says """ Do you think the above is all you should need? If so, you're right. You can stop reading now. If you think you need more, we've got that, too. Read on (you may need more coffee). """ No? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/10/2010 10:47 AM, Stephen J. Turnbull wrote: Vinay Sajip writes: > Indeed, and the very first code sample in the logging documentation > shows exactly the simplistic easy usage you're talking about. I > can't see why anyone would be scared off by that example. They're not scared by that example. What you need is a paragraph below it that says """ Do you think the above is all you should need? If so, you're right. You can stop reading now. If you think you need more, we've got that, too. Read on (you may need more coffee). """ +1 ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Fri, Dec 10, 2010 at 06:47:50PM +0900, Stephen J. Turnbull wrote: > Vinay Sajip writes: > > Indeed, and the very first code sample in the logging documentation > > shows exactly the simplistic easy usage you're talking about. I > > can't see why anyone would be scared off by that example. > > They're not scared by that example. What you need is a paragraph > below it that says > > """ > Do you think the above is all you should need? If so, you're > right. You can stop reading now. If you think you need more, > we've got that, too. Read on (you may need more coffee). > """ Better yet (IMHO) would be to split the huge page into "Logging: Simple start" and "Logging: Advanced usage (for the brave of of heart)". Oleg. -- Oleg Broytmanhttp://phd.pp.ru/[email protected] Programmers don't die, they just GOSUB without RETURN. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Terry Reedy udel.edu> writes: > Your proposal struck me as probably the best way forward. Can you code > it up and put a patch on the tracker that people can test before the > next beta? Coded up (including unit test) and checked into py3k branch, r87157. Regressions pass OK. Old behaviour can be obtained by setting logging.lastResort = None. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Hey Vinay, On Fri, 10 Dec 2010 11:46:18 + (UTC) Vinay Sajip wrote: > Terry Reedy udel.edu> writes: > > > Your proposal struck me as probably the best way forward. Can you code > > it up and put a patch on the tracker that people can test before the > > next beta? > > Coded up (including unit test) and checked into py3k branch, r87157. > Regressions > pass OK. Old behaviour can be obtained by setting logging.lastResort = None. When you make significant changes outside of logging, it would be nice if you first opened an issue and posted the patch for review. There's nothing obviously wrong with what you committed, but I think it's principally better to ask first for advice. (then there's nothing wrong with committing if you didn't get an opinion past a week or two) Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87157 - in python/branches/py3k: Lib/logging/__init__.py Lib/test/test_logging.py Misc/NEWS
On Fri, Dec 10, 2010 at 9:42 PM, vinay.sajip wrote: > Author: vinay.sajip > Date: Fri Dec 10 12:42:57 2010 > New Revision: 87157 > > Log: > logging: added handler of last resort. Nice, that ended up fitting in quite neatly. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Antoine Pitrou pitrou.net> writes: Hi Antoine, > When you make significant changes outside of logging, it would be nice > if you first opened an issue and posted the patch for review. There's > nothing obviously wrong with what you committed, but I think it's > principally better to ask first for advice. Sorry, which significant changes outside of logging do you mean? If you're referring to my addition of TestHandler and Matcher to test.support, I did mention this earlier in the thread, with a link to the code (though it wasn't on a patch added to the tracker, I agree). I can revert this change, if you want. The changes in r87157 are to Lib/logging/__init__.py, Lib/test/test_logging.py and Misc/NEWS - so IMO not outside of logging. Perhaps wrongly, I expected that these straightforward-seeming changes wouldn't adversely affect anyone else's workflow, except perhaps making it easier for Brian Quinlan to make changes to test_concurrent_futures to resolve #10626. I did run regressions and everything passed OK. > (then there's nothing wrong with committing if you didn't get an > opinion past a week or two) Sorry if my eagerness has stepped on any toes - it's not intentional. The diff of the changes is here: https://gist.github.com/736120 Comments are of course welcome. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Fri, 10 Dec 2010 12:34:48 + (UTC) Vinay Sajip wrote: > > When you make significant changes outside of logging, it would be nice > > if you first opened an issue and posted the patch for review. There's > > nothing obviously wrong with what you committed, but I think it's > > principally better to ask first for advice. > > Sorry, which significant changes outside of logging do you mean? If you're > referring to my addition of TestHandler and Matcher to test.support, I did > mention this earlier in the thread, with a link to the code (though it wasn't > on > a patch added to the tracker, I agree). I can revert this change, if you want. Yes, I am talking about this one. I am not asking that you revert it; I am just saying that it is generally appreciated if substantial patches get proposed on the tracker before being committed. In this case since it's in test.support we can probably improve it on the fly, of course. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] futures API
--- El jue, 9/12/10, Brian Quinlan escribió: > On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote: > > > I am looking forward to replacing a piece of code > > (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86) > by the futures module which was announced in python 3.2 > beta. I am a bit stuck with it, so I have a few questions > about the futures: > > > > 1. Is the futures API frozen? > > Yes. > > > 2. How hard would it be to return the tasks processed > in an output queue to process/consume the results while they > are returned? The code does not seem to be very open for > monkey patching. > > You can associate a callback with a submitted future. That > callback could add the future to your queue. Ok, it works. I was thinking the object was cleaned up immediately after it was used. > > 3. How hard would it be to add new tasks dynamically > (after a task is executed) and have the futures object never > complete? > > I'm not sure that I understand your question. You can > submit new work to an Executor at until time until it is > shutdown and a work item can take as long to complete as you > want. If you are contemplating tasks that don't complete > then maybe you could be better just scheduling a thread. > > > 4. Is there a performance evaluation of the futures > code (execution overhead) ? > > No. Scott Dial did make some performance improvements so he > might have a handle on its overhead. Ok. I have a process running for a long time, and which may use futures of different max_workers count. I think it is not too far-fetched to create a new futures object each time. Yet, the execution becomes slower after each call, for example with http://freehackers.org/~tnagy/futures_test.py: """ import concurrent.futures from queue import Queue import datetime class counter(object): def __init__(self, fut): self.fut = fut def run(self): def look_busy(num, obj): tot = 0 for x in range(num): tot += x obj.out_q.put(tot) start = datetime.datetime.utcnow() self.count = 0 self.out_q = Queue(0) for x in range(1000): self.count += 1 self.fut.submit(look_busy, self.count, self) while self.count: self.count -= 1 self.out_q.get() delta = datetime.datetime.utcnow() - start print(delta.total_seconds()) fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) for x in range(100): # comment the following line fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) c = counter(fut) c.run() """ The runtime grows after each step: 0.216451 0.225186 0.223725 0.74 0.230964 0.240531 0.24137 0.252393 0.249948 0.257153 ... Is there a mistake in this piece of code? Thanks, Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Antoine Pitrou pitrou.net> writes: > Yes, I am talking about this one. I am not asking that you revert it; I > am just saying that it is generally appreciated if substantial patches > get proposed on the tracker before being committed. > OK, sorry - point taken. > In this case since it's in test.support we can probably improve it on > the fly, of course. For sure. Regards, Vinay ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
Hi, Original discussion for the sysconfig module was short: http://mail.python.org/pipermail/python-dev/2009-May/089520.html Tarek will reply better, but I think the issue to solve was to move sysconfig out of distutils, improving its API a bit in the process but not overhauling it completely. A single function returning a nested dictionary could be added if deemed useful, but to me the existing API separates concerns (paths and variables for example) without being confusing nor having too many names. Exceptions: get_python_version could be removed; get_scheme_names and get_path_names could be replaced by constants. Maybe Tarek chose accessors to keep freedom for later, or just for consistency. sysconfig being new in 2.7 and 3.2, I don’t know how compatibility concerns apply. I think it would be bad to have highlighted the new module in 2.7 and then remove it in 3.2, since 2.6 and 2.7 have the goal of helping people moving to 3.x. Also, I don’t see where those 800 lines of code would live if the sysconfig module is removed; maybe in platform, which is already long? (it could be made smaller with some code modernization, removal of the popen function, but not much smaller). Final note: with 3.2 being in beta, I don’t know how much can be changed now. > The old way required source code available ability at runtime, the new > sysconfig module captures everything it needs at build time. Not yet :) See http://bugs.python.org/issue9878 Regards ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Oleg Broytman writes: >Better yet (IMHO) would be to split the huge page into "Logging: Simple > start" and "Logging: Advanced usage (for the brave of of heart)". Splitting is OK, but I disagree about the gloss "for the brave of heart". In my experience, if it is a YAGNI, the complexity is nearly impenetrable. If you *do* need it, it's not at all difficult to understand what the complexity is for, and it doesn't even look all that complex because it matches up with the problem you need to solve. If the documentation is still a deterrent, that's a problem with the documentation and it should be improved. AFAICT, making it clear that exporting all the internal flexibility is for those who need it, while most users will rarely need it, should be enough. But I'm not a good test case, since I already am familiar with XEmacs's similar system. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2010-12-03 - 2010-12-10) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open2557 (+20) closed 19899 (+38) total 22456 (+58) Open issues with patches: 1083 Issues opened (40) == #9101: reference json format in file formats chapter http://bugs.python.org/issue9101 reopened by eric.araujo #10553: Add optimize argument to builtin compile() and byte-compilatio http://bugs.python.org/issue10553 reopened by eric.araujo #10617: Collections ABCs canât be linked to http://bugs.python.org/issue10617 opened by eric.araujo #10618: regression in subprocess.call() command quoting http://bugs.python.org/issue10618 opened by djc #10619: Failed module loading in test discovery loses traceback http://bugs.python.org/issue10619 opened by michael.foord #10620: `python -m uniittest` should work with file paths as well as t http://bugs.python.org/issue10620 opened by michael.foord #10622: WebKit browsers show superfluous scrollbars in html docs http://bugs.python.org/issue10622 opened by davide.rizzo #10626: Bad interaction between test_logging and test_concurrent_futur http://bugs.python.org/issue10626 opened by pitrou #10627: Remove usage of deprecated configparser.ConfigParser class in http://bugs.python.org/issue10627 opened by lukasz.langa #10631: ZipFile and current directory change http://bugs.python.org/issue10631 opened by ocean-city #10632: multiprocessing generates a fatal error http://bugs.python.org/issue10632 opened by bquinlan #10634: Windows timezone changes not reflected by time.localtime http://bugs.python.org/issue10634 opened by eric.pruitt #10635: Calling subprocess.Popen with preexec_fn=signal.pause blocks f http://bugs.python.org/issue10635 opened by joseph.h.garvin #10636: subprocess module has race condition with SIGCHLD handlers http://bugs.python.org/issue10636 opened by joseph.h.garvin #10639: reindent.py converts newlines to platform default http://bugs.python.org/issue10639 opened by jaraco #10640: SystemError on pickling bytes >= 4GB http://bugs.python.org/issue10640 opened by hagen #10641: kill_python sometimes fails to kill processes on buildbots http://bugs.python.org/issue10641 opened by db3l #10642: site.py crashes on python startup due to defective .pth file http://bugs.python.org/issue10642 opened by dwr2 #10644: socket loses data, calling send()/sendall() on invalid socket http://bugs.python.org/issue10644 opened by diekmann #10645: egg-info file in lib-dynload http://bugs.python.org/issue10645 opened by ronaldoussoren #10646: ntpath.samefile doesn't work for hard links http://bugs.python.org/issue10646 opened by brian.curtin #10647: scrollbar crash in non-US locale format settings http://bugs.python.org/issue10647 opened by hfischer #10648: Extend peepholer to reverse loads or stores instead of build/u http://bugs.python.org/issue10648 opened by serprex #10650: decimal.py: quantize(): excess digits with watchexp=0 http://bugs.python.org/issue10650 opened by skrah #10652: test___all_ + test_tcl fails (Windows installed binary) http://bugs.python.org/issue10652 opened by ocean-city #10653: test_time test_strptime fails on windows http://bugs.python.org/issue10653 opened by ocean-city #10654: test_datetime fails on Python3.2 windows binary http://bugs.python.org/issue10654 opened by ocean-city #10655: Wrong powerpc define in Python/ceval.c http://bugs.python.org/issue10655 opened by adrian #10656: "Out of tree" build fails on AIX 5.3 http://bugs.python.org/issue10656 opened by pedz #10657: os.lstat/os.stat/os.fstat don't set st_dev (st_rdev) on Window http://bugs.python.org/issue10657 opened by ocean-city #10663: configure shouldn't set a default OPT http://bugs.python.org/issue10663 opened by pitrou #10664: xml.sax.expatreader should support namespace prefixes http://bugs.python.org/issue10664 opened by fdrake #10665: Update and expand unicodedata module documentation http://bugs.python.org/issue10665 opened by belopolsky #10666: OS X installer variants have confusing readline differences http://bugs.python.org/issue10666 opened by ned.deily #10667: collections.Counter object in C http://bugs.python.org/issue10667 opened by jpeel #10669: Remove Deprecation Warnings http://bugs.python.org/issue10669 opened by RusiMody #10670: Provide search scope limits http://bugs.python.org/issue10670 opened by techtonik #10671: urllib2 redirect to another host doesn't work http://bugs.python.org/issue10671 opened by kirax #10673: multiprocess.Process join method - timeout indistinguishable f http://bugs.python.org/issue10673 opened by Brian.Cain #10674: Unused dictmaker symbol in 2.* grammar http://bugs.python.org/issue10674 opened by salty-horse Most recent 15 issues with no replies (15) == #10674:
Re: [Python-Dev] futures API
On Fri, Dec 10, 2010 at 11:36 PM, Thomas Nagy wrote: > fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) > for x in range(100): > # comment the following line > fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) > c = counter(fut) > c.run() > """ > The runtime grows after each step: > Is there a mistake in this piece of code? This isn't an "overhead" question, it's a "how prompt is the resource release" question. Given that you've created circular references between the futures and your counter class, the answer is probably "not prompt at all". Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] futures API
Oops. I accidentally replied off-list: On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: --- El jue, 9/12/10, Brian Quinlan escribió: On Dec 9, 2010, at 4:26 AM, Thomas Nagy wrote: I am looking forward to replacing a piece of code (http://code.google.com/p/waf/source/browse/trunk/waflib/Runner.py#86 ) by the futures module which was announced in python 3.2 beta. I am a bit stuck with it, so I have a few questions about the futures: 1. Is the futures API frozen? Yes. 2. How hard would it be to return the tasks processed in an output queue to process/consume the results while they are returned? The code does not seem to be very open for monkey patching. You can associate a callback with a submitted future. That callback could add the future to your queue. Ok, it works. I was thinking the object was cleaned up immediately after it was used. 3. How hard would it be to add new tasks dynamically (after a task is executed) and have the futures object never complete? I'm not sure that I understand your question. You can submit new work to an Executor at until time until it is shutdown and a work item can take as long to complete as you want. If you are contemplating tasks that don't complete then maybe you could be better just scheduling a thread. 4. Is there a performance evaluation of the futures code (execution overhead) ? No. Scott Dial did make some performance improvements so he might have a handle on its overhead. Ok. I have a process running for a long time, and which may use futures of different max_workers count. I think it is not too far-fetched to create a new futures object each time. Yet, the execution becomes slower after each call, for example with http://freehackers.org/~tnagy/futures_test.py : """ import concurrent.futures from queue import Queue import datetime class counter(object): def __init__(self, fut): self.fut = fut def run(self): def look_busy(num, obj): tot = 0 for x in range(num): tot += x obj.out_q.put(tot) start = datetime.datetime.utcnow() self.count = 0 self.out_q = Queue(0) for x in range(1000): self.count += 1 self.fut.submit(look_busy, self.count, self) while self.count: self.count -= 1 self.out_q.get() delta = datetime.datetime.utcnow() - start print(delta.total_seconds()) fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) for x in range(100): # comment the following line fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) c = counter(fut) c.run() """ The runtime grows after each step: 0.216451 0.225186 0.223725 0.74 0.230964 0.240531 0.24137 0.252393 0.249948 0.257153 ... Is there a mistake in this piece of code? There is no mistake that I can see but I suspect that the circular references that you are building are causing the ThreadPoolExecutor to take a long time to be collected. Try adding: c = counter(fut) c.run() + fut.shutdown() Even if that fixes your problem, I still don't fully understand these numbers because I would expect the runtime to fall after a while as ThreadPoolExecutors are collected. Cheers, Brian Thanks, Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] transform() and untransform() methods, and the codec registry
On Thu, Dec 9, 2010 at 11:17 AM, Antoine Pitrou wrote:
> On Thu, 9 Dec 2010 13:55:08 -0500
> Alexander Belopolsky wrote:
>
>> On Thu, Dec 9, 2010 at 1:42 PM, Guido van Rossum wrote:
>> ..
>> > string-string transforms use the same namespace even though the
>> > typical transform only supports one or the other. E.g. IMO all of the
>> > following should raise LookupError:
>> >
>> b'abc'.transform('rot13')
>> > Traceback (most recent call last):
>> > ..
>> > return (input.translate(rot13_map), len(input))
>> > TypeError: expected an object with the buffer interface
>>
>> This is actually *very* misleading because
>>
>> >>> 'abc'.transform('rot13')
>> 'nop'
>>
>> works even though 'abc' is not "an object with the buffer interface".
>
> Agreed. It was already pointed out in the parent thread.
> I would say my opinion on keeping transform()/untransform() is +0 if
> these error messages (and preferably the exception type as well) get
> improved. Otherwise we'd better revert them and add a more polished
> version in 3.3.
Let me change my opinion then and say that I'm -1 on keeping the
confusing situation. It can be fixed either by postponing transform()
until 3.3 or by fixing the error checking and namespace issue.
Sorry about the reversal of pronouncements, but my original approval
was very weak indeed and mostly driven by "stick to status quo unless
there's a serious problem". The followup made it clear that the errors
and namespace issue are more seriously than I estimated at first.
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Glenn Linderman wrote:
> On 12/10/2010 12:06 AM, Vinay Sajip wrote:
> >> > This simplistic easy usage somewhat echo's Glenn's comment on this
> >> > thread
> > about logging seeming way to daunting as presented today. It needn't be.
> >> >
> > Indeed, and the very first code sample in the logging documentation shows
> > exactly the simplistic easy usage you're talking about. I can't see why
> > anyone
> > would be scared off by that example.
>
> It didn't scare me off. But it wasn't _quite_ comprehensive enough to
> keep me from being put off by the next bit of manual, (about 8%, as
> mentioned before), which had lots of discussion, but no more examples,
> so what is lacking up front is a few more examples of using the
> convenience methods as Greg apparently does, without configuration,
> but showing use in multiple modules. After it is shown to be simple
> to get started in a multi-module code base, then all the refinements
> can be explained and added, but by then people are already using the
> logger, and learning becomes incremental.
Just a note: after spacing out over this thread for a bit, I thought I'd
try out logging, which I'd never tried before, in some code I was
writing. I added a couple of lines like
logging.info("foo is %s", foo)
where I'd usually write
sys.stderr.write("foo is %s\n" % foo)
Liked it just fine -- easier to write. But nothing came out! Because
it's got this odd idea (from Java?) about "severities" of messages, and
by default passes nothing less "threatening" than warnings, because
logger.root is initialized by default to WARNING.
So, still a bit disconcerting for newbies.
Bill
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] futures API
--- El vie, 10/12/10, Brian Quinlan escribió: > On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: > > I have a process running for a long time, and which > may use futures of different max_workers count. I think it > is not too far-fetched to create a new futures object each > time. Yet, the execution becomes slower after each call, for > example with http://freehackers.org/~tnagy/futures_test.py: > > > > """ > > import concurrent.futures > > from queue import Queue > > import datetime > > > > class counter(object): > > def __init__(self, fut): > > self.fut = fut > > > > def run(self): > > def > look_busy(num, obj): > > > tot = 0 > > > for x in range(num): > > > tot += x > > > obj.out_q.put(tot) > > > > start = > datetime.datetime.utcnow() > > self.count = 0 > > self.out_q = > Queue(0) > > for x in > range(1000): > > > self.count += 1 > > > self.fut.submit(look_busy, self.count, > self) > > > > while > self.count: > > > self.count -= 1 > > > self.out_q.get() > > > > delta = > datetime.datetime.utcnow() - start > > > print(delta.total_seconds()) > > > > fut = > concurrent.futures.ThreadPoolExecutor(max_workers=20) > > for x in range(100): > > # comment the following line > > fut = > concurrent.futures.ThreadPoolExecutor(max_workers=20) > > c = counter(fut) > > c.run() > > """ > > > > The runtime grows after each step: > > 0.216451 > > 0.225186 > > 0.223725 > > 0.74 > > 0.230964 > > 0.240531 > > 0.24137 > > 0.252393 > > 0.249948 > > 0.257153 > > ... > > > > Is there a mistake in this piece of code? > > There is no mistake that I can see but I suspect that the > circular references that you are building are causing the > ThreadPoolExecutor to take a long time to be collected. Try > adding: > > c = counter(fut) > c.run() > + fut.shutdown() > > Even if that fixes your problem, I still don't fully > understand this because I would expect the runtime to fall > after a while as ThreadPoolExecutors are collected. The shutdown call is indeed a good fix :-) Here is the time response of the calls to counter() when shutdown is not called: http://www.freehackers.org/~tnagy/runtime_futures.png After trying to stop the program by using CTRL+C, the following error may appear, after which the process cannot be interrupted: """ 19:18:12 /tmp/build> python3.2 futures_test.py 0.389657 0.417173 0.416513 0.421424 0.449666 0.482273 ^CTraceback (most recent call last): File "futures_test.py", line 36, in c.run() File "futures_test.py", line 22, in run self.fut.submit(look_busy, self.count, self) File "/usr/local/lib/python3.2/concurrent/futures/thread.py", line 114, in submit self._work_queue.put(w) File "/usr/local/lib/python3.2/queue.py", line 135, in put self.not_full.acquire() KeyboardInterrupt """ It is not expected, is it? Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] futures API
On Dec 10, 2010, at 10:51 AM, Thomas Nagy wrote: --- El vie, 10/12/10, Brian Quinlan escribió: On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: I have a process running for a long time, and which may use futures of different max_workers count. I think it is not too far-fetched to create a new futures object each time. Yet, the execution becomes slower after each call, for example with http://freehackers.org/~tnagy/futures_test.py: """ import concurrent.futures from queue import Queue import datetime class counter(object): def __init__(self, fut): self.fut = fut def run(self): def look_busy(num, obj): tot = 0 for x in range(num): tot += x obj.out_q.put(tot) start = datetime.datetime.utcnow() self.count = 0 self.out_q = Queue(0) for x in range(1000): self.count += 1 self.fut.submit(look_busy, self.count, self) while self.count: self.count -= 1 self.out_q.get() delta = datetime.datetime.utcnow() - start print(delta.total_seconds()) fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) for x in range(100): # comment the following line fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) c = counter(fut) c.run() """ The runtime grows after each step: 0.216451 0.225186 0.223725 0.74 0.230964 0.240531 0.24137 0.252393 0.249948 0.257153 ... Is there a mistake in this piece of code? There is no mistake that I can see but I suspect that the circular references that you are building are causing the ThreadPoolExecutor to take a long time to be collected. Try adding: c = counter(fut) c.run() +fut.shutdown() Even if that fixes your problem, I still don't fully understand this because I would expect the runtime to fall after a while as ThreadPoolExecutors are collected. The shutdown call is indeed a good fix :-) Here is the time response of the calls to counter() when shutdown is not called: http://www.freehackers.org/~tnagy/runtime_futures.png FWIW, I think that you are confusion the term "future" with "executor". A future represents a single work item. An executor creates futures and schedules their underlying work. Hmmmthat is very suspicious - it looks like the ThreadPoolExecutors are not being collected. If you are feeling bored you could figure out why not :-) After trying to stop the program by using CTRL+C, the following error may appear, after which the process cannot be interrupted: """ 19:18:12 /tmp/build> python3.2 futures_test.py 0.389657 0.417173 0.416513 0.421424 0.449666 0.482273 ^CTraceback (most recent call last): File "futures_test.py", line 36, in c.run() File "futures_test.py", line 22, in run self.fut.submit(look_busy, self.count, self) File "/usr/local/lib/python3.2/concurrent/futures/thread.py", line 114, in submit self._work_queue.put(w) File "/usr/local/lib/python3.2/queue.py", line 135, in put self.not_full.acquire() KeyboardInterrupt """ It is not expected, is it? It isn't surprising. Python lock acquisitions are not interruptible and anytime you interrupt a program that manipulates locks you may kill the code that was going to cause the lock to be released. Cheers, Brian Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Bill Janssen parc.com> writes: > Liked it just fine -- easier to write. But nothing came out! Because > it's got this odd idea (from Java?) about "severities" of messages, and > by default passes nothing less "threatening" than warnings, because > logger.root is initialized by default to WARNING. The idea about severity of messages is based on the premise that there might be different audiences for logged messages, and that these can change over time. A fairly common developer perspective on logging is that a logger equates to a destination (e.g. console or file) and the only audience for logging is the developer themselves. So, "handlers" and "loggers" seem complicated, perhaps over-complicated. In more complex systems, there are different stakeholders: end users, sys admins, support teams, development teams. You have to consider "what happened" (logging event), "where it happened" (logger name), "how important it is" (severity or level) and "who wants to know" (handler), because not everyone wants to know about everything all the time. That's pretty independent of which language you're developing in; it's more a function of the complexity of the system you're developing. Since Python is used in some fairly large system scenarios - NASA, Google, etc. - it doesn't make sense not to have the whole enchilada of logging functionality, at the risk of making it seem complicated to users with less complex needs. And importance maps reasonably naturally onto an idea of integer levels. The default setting of WARNING is not from Java either - it's the Unix idea that verbosity should be kept to a minimum unless specifically requested. So, warnings and errors should be shown, but info and debug not. Imagine if the default level was DEBUG - lots of detailed stuff would be displayed, which would be off-putting for most users. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] futures API
--- El vie, 10/12/10, Thomas Nagy escribió: > --- El vie, 10/12/10, Brian Quinlan > escribió: > > On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: > > > I have a process running for a long time, and > which > > may use futures of different max_workers count. I > think it > > is not too far-fetched to create a new futures object > each > > time. Yet, the execution becomes slower after each > call, for > > example with http://freehackers.org/~tnagy/futures_test.py: > > > > > > """ > > > import concurrent.futures > > > from queue import Queue > > > import datetime > > > > > > class counter(object): > > > def __init__(self, fut): > > > self.fut = fut > > > > > > def run(self): > > > def > > look_busy(num, obj): > > > > > tot = 0 > > > > > for x in range(num): > > > > > tot += x > > > > > obj.out_q.put(tot) > > > > > > start = > > datetime.datetime.utcnow() > > > self.count = 0 > > > self.out_q = > > Queue(0) > > > for x in > > range(1000): > > > > > self.count += 1 > > > > > self.fut.submit(look_busy, self.count, > > self) > > > > > > while > > self.count: > > > > > self.count -= 1 > > > > > self.out_q.get() > > > > > > delta = > > datetime.datetime.utcnow() - start > > > > > print(delta.total_seconds()) > > > > > > fut = > > concurrent.futures.ThreadPoolExecutor(max_workers=20) > > > for x in range(100): > > > # comment the following line > > > fut = > > concurrent.futures.ThreadPoolExecutor(max_workers=20) > > > c = counter(fut) > > > c.run() > > > """ > > > > > > The runtime grows after each step: > > > 0.216451 > > > 0.225186 > > > 0.223725 > > > 0.74 > > > 0.230964 > > > 0.240531 > > > 0.24137 > > > 0.252393 > > > 0.249948 > > > 0.257153 > > > ... > > > > > > Is there a mistake in this piece of code? > > > > There is no mistake that I can see but I suspect that > the > > circular references that you are building are causing > the > > ThreadPoolExecutor to take a long time to be > collected. Try > > adding: > > > > c = counter(fut) > > c.run() > > + fut.shutdown() > > > > Even if that fixes your problem, I still don't fully > > understand this because I would expect the runtime to > fall > > after a while as ThreadPoolExecutors are collected. > > The shutdown call is indeed a good fix :-) Here is the time > response of the calls to counter() when shutdown is not > called: > http://www.freehackers.org/~tnagy/runtime_futures.png > > After trying to stop the program by using CTRL+C, the > following error may appear, after which the process cannot > be interrupted: > > """ > 19:18:12 /tmp/build> python3.2 futures_test.py > 0.389657 > 0.417173 > 0.416513 > 0.421424 > 0.449666 > 0.482273 > ^CTraceback (most recent call last): > File "futures_test.py", line 36, in > c.run() > File "futures_test.py", line 22, in run > self.fut.submit(look_busy, self.count, self) > File > "/usr/local/lib/python3.2/concurrent/futures/thread.py", > line 114, in submit > self._work_queue.put(w) > File "/usr/local/lib/python3.2/queue.py", line 135, in > put > self.not_full.acquire() > KeyboardInterrupt > """ > > It is not expected, is it? The problem also occurs when using a callback: http://www.freehackers.org/~tnagy/futures_test2.py If it is necessary to catch KeyboardInterrupt exceptions to cancel the futures execution, then how about adding this detail to the docs? Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] futures API
On Dec 10, 2010, at 11:39 AM, Thomas Nagy wrote: --- El vie, 10/12/10, Thomas Nagy escribió: --- El vie, 10/12/10, Brian Quinlan escribió: On Dec 10, 2010, at 5:36 AM, Thomas Nagy wrote: I have a process running for a long time, and which may use futures of different max_workers count. I think it is not too far-fetched to create a new futures object each time. Yet, the execution becomes slower after each call, for example with http://freehackers.org/~tnagy/futures_test.py: """ import concurrent.futures from queue import Queue import datetime class counter(object): def __init__(self, fut): self.fut = fut def run(self): def look_busy(num, obj): tot = 0 for x in range(num): tot += x obj.out_q.put(tot) start = datetime.datetime.utcnow() self.count = 0 self.out_q = Queue(0) for x in range(1000): self.count += 1 self.fut.submit(look_busy, self.count, self) while self.count: self.count -= 1 self.out_q.get() delta = datetime.datetime.utcnow() - start print(delta.total_seconds()) fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) for x in range(100): # comment the following line fut = concurrent.futures.ThreadPoolExecutor(max_workers=20) c = counter(fut) c.run() """ The runtime grows after each step: 0.216451 0.225186 0.223725 0.74 0.230964 0.240531 0.24137 0.252393 0.249948 0.257153 ... Is there a mistake in this piece of code? There is no mistake that I can see but I suspect that the circular references that you are building are causing the ThreadPoolExecutor to take a long time to be collected. Try adding: c = counter(fut) c.run() +fut.shutdown() Even if that fixes your problem, I still don't fully understand this because I would expect the runtime to fall after a while as ThreadPoolExecutors are collected. The shutdown call is indeed a good fix :-) Here is the time response of the calls to counter() when shutdown is not called: http://www.freehackers.org/~tnagy/runtime_futures.png After trying to stop the program by using CTRL+C, the following error may appear, after which the process cannot be interrupted: """ 19:18:12 /tmp/build> python3.2 futures_test.py 0.389657 0.417173 0.416513 0.421424 0.449666 0.482273 ^CTraceback (most recent call last): File "futures_test.py", line 36, in c.run() File "futures_test.py", line 22, in run self.fut.submit(look_busy, self.count, self) File "/usr/local/lib/python3.2/concurrent/futures/thread.py", line 114, in submit self._work_queue.put(w) File "/usr/local/lib/python3.2/queue.py", line 135, in put self.not_full.acquire() KeyboardInterrupt """ It is not expected, is it? The problem also occurs when using a callback: http://www.freehackers.org/~tnagy/futures_test2.py If it is necessary to catch KeyboardInterrupt exceptions to cancel the futures execution, then how about adding this detail to the docs? AFAIK, catching KeyboardInterrupt exceptions is not sufficient. Cheers, Brian Thomas ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/brian%40sweetapp.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
On Dec 10, 2010, at 6:20 AM, Éric Araujo wrote:
> Final note: with 3.2 being in beta, I don’t know how much can be changed
> now.
Part of the purpose of a beta, and in our case, two betas is to give
people a chance to exercise new APIs and fix them before they
become set in stone two months later.
IMO, sysconfig did not warrant a whole module. The pile of
awkward accessor functions is harder to learn / remember
than the simple data structure shown in the last email.
Rather than using two levels of dictionary, it's also possible
to use a named tuple if you think that is more clean looking:
>>> c = sys.sysconfig()
>>> c.config_vars.get('SO)
'.pyd'
>>> c.platform
'win32'
>>> c.paths.get('stdlib')
'C:\\Python32\\Lib'
>>> # the named tuple fields:
>>> c.fields
['config_vars', 'platform', 'version', 'scheme_names', 'paths']
This would be a straight-forward API that uses existing, well-known
tools (attribute access and dict.get) instead of an unnecessary
nest of ad-hoc accessor functions living in a new module.
Tastes may vary on writing getter functions for every category
of interest but we did not need a whole new module for this.
The referenced email didn't indicate much if any thought about
the API design, so I think should get that thought now.
If those functions were already public in distutils, it is trivial
to write an adapter for them there.
Raymond
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Fri, Dec 10, 2010 at 11:39 AM, Vinay Sajip wrote: > Bill Janssen parc.com> writes: > >> Liked it just fine -- easier to write. But nothing came out! Because >> it's got this odd idea (from Java?) about "severities" of messages, and >> by default passes nothing less "threatening" than warnings, because >> logger.root is initialized by default to WARNING. > > The idea about severity of messages is based on the premise that there might > be > different audiences for logged messages, and that these can change over time. > A > fairly common developer perspective on logging is that a logger equates to a > destination (e.g. console or file) and the only audience for logging is the > developer themselves. So, "handlers" and "loggers" seem complicated, perhaps > over-complicated. > > In more complex systems, there are different stakeholders: end users, sys > admins, support teams, development teams. You have to consider "what happened" > (logging event), "where it happened" (logger name), "how important it is" > (severity or level) and "who wants to know" (handler), because not everyone > wants to know about everything all the time. That's pretty independent of > which > language you're developing in; it's more a function of the complexity of the > system you're developing. Since Python is used in some fairly large system > scenarios - NASA, Google, etc. - it doesn't make sense not to have the whole > enchilada of logging functionality, at the risk of making it seem complicated > to > users with less complex needs. And importance maps reasonably naturally onto > an > idea of integer levels. > > The default setting of WARNING is not from Java either - it's the Unix idea > that > verbosity should be kept to a minimum unless specifically requested. So, > warnings and errors should be shown, but info and debug not. Imagine if the > default level was DEBUG - lots of detailed stuff would be displayed, which > would > be off-putting for most users. And yet, I have helped many people who were baffled by exactly what Bill observed: logging.info() didn't do anything. Maybe the default should be INFO? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Fri, 10 Dec 2010 12:31:19 -0800 Guido van Rossum wrote: > > The default setting of WARNING is not from Java either - it's the Unix idea > > that > > verbosity should be kept to a minimum unless specifically requested. So, > > warnings and errors should be shown, but info and debug not. Imagine if the > > default level was DEBUG - lots of detailed stuff would be displayed, which > > would > > be off-putting for most users. > > And yet, I have helped many people who were baffled by exactly what > Bill observed: logging.info() didn't do anything. Maybe the default > should be INFO? Funny, because displaying only errors and silencing other messages is exactly what I expected logging to do by default. Regards Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
On Fri, 10 Dec 2010 12:27:26 -0800
Raymond Hettinger wrote:
>
> IMO, sysconfig did not warrant a whole module.
Where would you put it?
> Rather than using two levels of dictionary, it's also possible
> to use a named tuple if you think that is more clean looking:
>
>>>> c = sys.sysconfig()
>>>> c.config_vars.get('SO)
>'.pyd'
Some of these things are computed at runtime by parsing makefiles and
other stuff. You don't want to do it as soon as the module is imported.
There may also be some bootstrap issues when importing some of the
necessary modules from the top level.
The current API is fine IMO. Perhaps it could be refined a bit, but
there's nothing horrendous. It's not for everyone to use; creating
dict-like classes over these APIs would be a terrible waste of time,
and add a useless maintenance burden.
> If those functions were already public in distutils, it is trivial
> to write an adapter for them there.
The whole point was to separate them from distutils.
Regards
Antoine.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/10/2010 12:49 PM, Antoine Pitrou wrote: And yet, I have helped many people who were baffled by exactly what > Bill observed: logging.info() didn't do anything. Maybe the default > should be INFO? Funny, because displaying only errors and silencing other messages is exactly what I expected logging to do by default. So we are slowly learning the things that should be on the first couple pages of the logging docs... 1) simple example for one file programs, include an example of specifying output severity threshold. I'm with Antoine here on my expectations. 2) example for multi-module, showing how a single logging destination causes logging to happen in all modules, at the same level (if that is the case, which I hope it is). 3) Maybe a small discussion of log formatting should be next? So the user realizes he shouldn't do the message formatting himself? 4) Then rotating logs for long-running programs. And then the rest of the more feature-filled stuff. I agree logging needs to be full-featured, but many of the features are only necessary in complex environments, and simple stuff should be simple, if it can be, and those cases explained first. Hopefully a useful subset for simple cases can be described in 1-3 pages, and then the complex features after that. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Guido van Rossum wrote: > And yet, I have helped many people who were baffled by exactly what > Bill observed: logging.info() didn't do anything. Maybe the default > should be INFO? Yeah, I was curious enough to read the code and find out why. But many won't. By the way, I tried reading the 2.7 docs first, and didn't figure it out from them. So I looked at the code. Bill ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Glenn Linderman wrote: > 1) simple example for one file programs, include an example of > specifying output severity threshold. I'm with Antoine here on my > expectations. Yes, once I put logging.basicConfig(stream=sys.stdout, level=logging.DEBUG) in my main(), I got what I thought I'd get in the first place. All logging messages. But I can see Antoine's point: if we use this in the stdlib, you don't want that default. I think that logging events (context) have to come into this; you can't do it with just severity alone. I'd expect to have different settings, by default, for __main__ and for http.client, for instance. Bill ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
On Dec 10, 2010, at 12:56 PM, Antoine Pitrou wrote:
> On Fri, 10 Dec 2010 12:27:26 -0800
> Raymond Hettinger wrote:
>>
>> IMO, sysconfig did not warrant a whole module.
>
> Where would you put it?
A single function in the sys module.
>
>> Rather than using two levels of dictionary, it's also possible
>> to use a named tuple if you think that is more clean looking:
>>
> c = sys.sysconfig()
> c.config_vars.get('SO)
>> '.pyd'
>
> Some of these things are computed at runtime by parsing makefiles and
> other stuff. You don't want to do it as soon as the module is imported.
The proposal is for a function that does the computation when invoked,
not when imported. The function returns a named tuple so that we can
use Python's builtin accessors like attributes and dict.get().
Raymond
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
On Thu, 09 Dec 2010 16:18:14 -0800, Raymond Hettinger wrote: > Does anyone know why this needed a separate module and so many accessor > functions? Originally sysconfig was moved *out* of distutils, and distutils was changed to use it. But that proved to be as fragile as many other distutils changes. When distutils was reverted, sysconfig was kept because of the long term desire to move it out of distutils. At that point I think it would have been very easy to propose API changes, but I suppose it didn't occur to anyone to think about that possibility. > ISTM it mostly could have been reduced to single call returning a nested > dictionary. If what was returned was, as you suggested on IRC, a set of named tuples, it seems to me that would be a very convenient interface. And since all values (as I understand it) are intended to be static (even if some aren't at the moment), it seems to make sense from an implementation standpoint as well. Like Ãric, I'm not sure what the implications of the existing module having been released in 2.7 and 3.2 beta are in terms of making such an API change. -- R. David Murray www.bitdance.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Antoine Pitrou pitrou.net> writes: > Guido van Rossum python.org> wrote: > > And yet, I have helped many people who were baffled by exactly what > > Bill observed: logging.info() didn't do anything. Maybe the default > > should be INFO? > > Funny, because displaying only errors and silencing other messages is > exactly what I expected logging to do by default. The one thing I'm sure of is that it's hard to find more than a couple of people with the same view about what the default should be. Anyone for bikesheds? :-) IMO as long as it's just a small amount of work to get the specific effect that you want, it doesn't really matter too much what the default is - though of course I'd like it to be "right", whatever that means ;-) And whatever it is, it's going to upset *some* group of people because they have to add one line (or so) to get it to do what they want, and they feel really strongly that they shouldn't have to add that one line. In a way, I suppose I'm not unhappy that this is all we're arguing about and not anything more substantive. Some useful feedback about the documentation has emerged, too. I would (naturally) like to see broader adoption of logging in the standard library, and the view from Python core developers who have expressed opinions on this thread seems to be that the present behaviour is an obstacle to that adoption. I have my views but I'm not hidebound by them, hence the proposed change and the posting about it on the logging blog and c.l.py, just to let people know what's coming. It'll be interesting to see, once this change is implemented, whether the suggested obstacle really has been an obstacle, rather than (as I suspect) the low level of logging adoption in the stdlib being largely being a matter of plain inertia. But even if it helps make possible a better resolution of #10626 (as an archetype of unraisable-exception scenarios), that would be a good outcome. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Bill Janssen parc.com> writes: > stdlib, you don't want that default. I think that logging events > (context) have to come into this; you can't do it with just severity > alone. I'd expect to have different settings, by default, for __main__ > and for http.client, for instance. > Although e.g. http.client may set a default severity, the developer integrating it into an application is free to override that severity for the purposes of their application. Typically, there are different severities set for different parts of an application at different times, depending on where the focus of diagnostic scrutiny is at those times. Severity is not the only filtering mechanism, merely the simplest and most common one, that people can readily grasp. Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] futures API
On Sat, Dec 11, 2010 at 6:07 AM, Brian Quinlan wrote: >> The problem also occurs when using a callback: >> http://www.freehackers.org/~tnagy/futures_test2.py >> >> If it is necessary to catch KeyboardInterrupt exceptions to cancel the >> futures execution, then how about adding this detail to the docs? > > AFAIK, catching KeyboardInterrupt exceptions is not sufficient. finally blocks and with statements can get you a fairly long way, though. futures does everything right on this front, as far as I can see. In this case, the problem is in the design of the test program. It *must* get exactly as many items in the queue as were submitted to the executor. If one is ever missing (e.g. due to a poorly timed KeyboardInterrupt in the callback, as clearly happened in the presented trace), it will hang in the final call to self.out_q.get(). There's a reason Queue.get offers both nonblocking and block-with-timeout functionality. (Alternately, the management of out_q and count could be made smarter, such that an exception in adding a result to out_q triggered an appropriate adjustment in the count of expected values) I also agree with Brian that the variable naming for the sample code and comments like "may use futures of different max_workers count. I think it is not too far-fetched to create a new futures object each time" suggests a fundamental confusion of the terms "future" and "executor". Executors are not futures - the return value from an executor's submit method is a future. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] (Non-) use of weakref.WeakSet in concurrent.futures (was Re: futures API)
On Sat, Dec 11, 2010 at 6:07 AM, Brian Quinlan wrote: > AFAIK, catching KeyboardInterrupt exceptions is not sufficient. Getting away from the flaws in the test code in this thread, is there any particular reason futures is rolling its own variant of weakref.WeakSet for the _thread_references cleanup sets? It seems to me that using WeakSet would mean you could get rid of _remove_dead_thread_references altogether (the implicit callbacks would handle that part), and then a set() call in _python_exit would give you concrete references to work with for cleanup purposes. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Glenn Linderman writes: > 1) simple example for one file programs, include an example of > specifying output severity threshold. I'm with Antoine here on my > expectations. > > 2) example for multi-module, showing how a single logging destination > causes logging to happen in all modules, at the same level (if that is > the case, which I hope it is). > > 3) Maybe a small discussion of log formatting should be next? So the > user realizes he shouldn't do the message formatting himself? As advice, this shouldn't need to be more than a couple of lines, with a two line example, right? But it does deserve a prominent header. > 4) Then rotating logs for long-running programs. > > And then the rest of the more feature-filled stuff. +1 for this agenda. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/10/2010 5:16 PM, Vinay Sajip wrote: IMO as long as it's just a small amount of work to get the specific effect that you want, it doesn't really matter too much what the default is - though of course I'd like it to be "right", whatever that means ;-) I think the default should accomodate both internal uses (stdlib and elsewhere) and beginner uses. I suspect the latter would mean outputting most everything, to the convenience functions work (as a beginner might expect). -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
On 12/10/2010 4:59 PM, R. David Murray wrote: Like Éric, I'm not sure what the implications of the existing module having been released in 2.7 and 3.2 beta are in terms of making such an API change. I am with Raymond on this: the purpose of betas is so we can test *and* make changes. No one should base production software on a beta release. -- Terry Jan Reedy ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
On Sat, Dec 11, 2010 at 7:59 AM, R. David Murray wrote: > On Thu, 09 Dec 2010 16:18:14 -0800, Raymond Hettinger > wrote: >> ISTM it mostly could have been reduced to single call returning a nested >> dictionary. > > If what was returned was, as you suggested on IRC, a set of named tuples, > it seems to me that would be a very convenient interface. And since > all values (as I understand it) are intended to be static (even if some > aren't at the moment), it seems to make sense from an implementation > standpoint as well. > > Like Éric, I'm not sure what the implications of the existing module > having been released in 2.7 and 3.2 beta are in terms of making such an > API change. We've reverted from beta to pseudo-alpha after discovering sufficiently egregious mistakes in the past. (we didn't *actually* revert the build naming because release IDs that don't increase monotonically can confuse matters). However, I don't think this is sufficiently egregious to qualify. My own reasons for advocating retention of the separate module: 1. Something equivalent to the current sysconfig module is going to be needed to actually work out the contents of the data structure. Rewriting all that in C would be silly, so the most that could happen is to rename "sysconfig" to "_sysconfig". 2. Invoking Python code from sys module functions is possible but untidy (generally involving implicit module imports). Since we need a module anyway for implementation purposes, why not make it public? 3. The sysconfig module docs provide a nicer space to talk about the purpose of the provided information rather than squirreling it away in a corner of the sys module docs 4. "python -m sysconfig" provides a useful dump of the system status 5. The API design did not get much *new* thought, as the idea was to allow users to do anything that the old distutils.sysconfig could do, but without requiring them to depend on distutils to get that functionality. The easiest way to ensure that is to just copy the existing API rather than trying to get creative. Don't think of it as adding a "whole new module", thinking of it as decoupling a relatively small existing module (i.e. distutils.sysconfig) from a larger package that isn't installed by default on all systems (i.e. distutils). A concrete problem with Raymond's idea in particular is that there is actually a full set of paths defined for *each scheme*. get_path() and get_paths() let you ignore this most of the time, since there is an appropriate default scheme defined for each platform, so they just use that if you don't specify one explicitly. A named tuple could obviously expose the paths for all of the schemes, with the paths for the default scheme duplicated at the top level, but the data structure starts looking very messy at that point. The named tuple idea also conflates meta-data about the sysconfig data (the list of scheme names and path names) with the actual data for the current platform (platform, version, paths, config_vars). So I think reusing the tried and tested API is easily justified in this case. What may be desirable in 3.3 (particularly with the distutils2 reintegration taking place), is to revisit the sysconfig API to see if there are better ways to expose some of the data (such as a function returning a named tuple containing the raw data underlying the output of "python -m sysconfig"). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for the new sysconfig module
On Sat, Dec 11, 2010 at 12:44 PM, Terry Reedy wrote: > On 12/10/2010 4:59 PM, R. David Murray wrote: > >> Like Éric, I'm not sure what the implications of the existing module >> having been released in 2.7 and 3.2 beta are in terms of making such an >> API change. > > I am with Raymond on this: the purpose of betas is so we can test *and* make > changes. No one should base production software on a beta release. It's also the key difference between betas and release candidates. Completely new features are still out after the first beta, but tweaking the APIs of new-in-this-release features is still possible with sufficient justification. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Stephen J. Turnbull wrote: They're not scared by that example. What you need is a paragraph below it that says """ Do you think the above is all you should need? If so, you're right. You can stop reading now. If you think you need more, we've got that, too. Read on (you may need more coffee). """ +1 Oleg Broytman writes: >Better yet (IMHO) would be to split the huge page into "Logging: Simple > start" and "Logging: Advanced usage (for the brave of of heart)". Splitting is OK, but I disagree about the gloss "for the brave of heart". In my experience, if it is a YAGNI, the complexity is nearly impenetrable. If you *do* need it, it's not at all difficult to understand what the complexity is for, and it doesn't even look all that complex because it matches up with the problem you need to solve. If the documentation is still a deterrent, that's a problem with the documentation and it should be improved. AFAICT, making it clear that exporting all the internal flexibility is for those who need it, while most users will rarely need it, should be enough. But I'm not a good test case, since I already am familiar with XEmacs's similar system. I think I'm a pretty good test case -- knew nothing about logging, still don't know much, found documentation comprehensive but unweildy, and would *still* benefit from a more extensive (though still short) beginner's section, with the prominent paragraph stating I now know enough for simple cases. :) Oh, and awesome module, by the way. Thank you. ~Ethan~ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Sat, Dec 11, 2010 at 12:42 PM, Terry Reedy wrote: > On 12/10/2010 5:16 PM, Vinay Sajip wrote: > >> IMO as long as it's just a small amount of work to get the specific effect >> that you want, it doesn't really matter too much what the default is - >> though >> of course I'd like it to be "right", whatever that means ;-) > > I think the default should accomodate both internal uses (stdlib and > elsewhere) and beginner uses. I suspect the latter would mean outputting > most everything, to the convenience functions work (as a beginner might > expect). I think the main point of confusion is going to be what happens to .info() messages by default. With Vinay's latest changes, the 3.2 default is going to be: debug() -> /dev/null info() -> /dev/null warning() -> stderr error() -> stderr critical() -> stderr exception() -> stderr This will apply both to the convenience functions (where this is already the case due to the implicit basicConfig() call and the default level setting on the root logger) and to logger methods (where the new "handler of last resort" will apply instead of emitting the comparatively unhelpful "no handler found" error message). This is the correct behaviour for library code and even many applications - silent when all goes well, but if anything goes wrong that can't be flagged with an exception, the errors are displayed by default. While Vinay is right that inertia plays a large part in not using the logging module in the standard library, the new behaviour for direct calls to loggers *is* a direct result of a problem encountered in the concurrent.futures tests due to the fact that Brian previously couldn't rely on an unconfigured logging module doing the right thing. However, the confusion that this setup will engender is that encountered by Bill: by default, info() messages are silenced rather than displayed on stdout. Notice that even the recommended "basicConfig" approach to resolving this is subtly flawed, since your choices are to either display info messages on stderr (by specifying just a level argument) or to redirect warning() et al to stdout instead of stderr (by also specifying a stream argument). The "correct" answer (info and below to stdout, warning and above to stderr) is actually quite tricky to set up, since handlers have no innate concept of a "maximum level", so you have to provide a filter function (more on that below [1]). Basically, as of 3.2, the correct "default" use is likely to be: print() for information you want to appear on stdout by default (especially in scripts) logging.debug() and .info() for additional debugging information that should be silenced by default warnings.warn() for actual programmatic warnings that should obey warning filter rules (logging can hook the warnings display mechanism) logging.warn() for warnings that should never raise an exception and should be emitted every time they are encountered raise an exception for normal error reporting logging.error(), .critical() and .exception() for error reporting that cannot raise an exception for some reason (e.g. long running processes that are simply recording that an exception occurred) Libraries can now safely use their own getLogger(__name__) loggers for the latter two uses without worrying about the output being lost by the default logging configuration, so they become a much better option than printing directly to sys.stderr. Most importantly, someone debugging the app will be able to insert additional context information (e.g. stack traces, local variables values) without needing to alter the code emitting the error message. Regards, Nick. [1] As an exercise, I decided to actually configure recommended console logging output style via the logging module. It proved to be quite informative as to where some of the flaws in the current documentation lie. First and foremost, it was genuinely hard for me to figure out that one of my key problems was actually that the root logger is created with a level of WARNING by default so my handlers weren't even being invoked. The only place I found this very important detail was in the documentation of Logger.setLevel()! There really should be a section early on in the logging documentation describing the root logger, how to use it for basic logging, and its default settings (with and without a call to "basicConfig"). Such a section would then include a link to the latter section on the logging heirarchy. In addition to that, the documentation of the LogRecord class should really include a table of attribute names that are available for use in formatting and filtering. Initially I thought the attribute names corresponded to the parameter names, and subsequently resorted to a dir() call in my filter to figure out what the actual names were after "lvl" caused an AttributeError. Armed with the fact that the attribute I wanted was called "levelno", I was able to discover the list of LogRecord attributes was actually in the *Formatter* documen
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On Sat, Dec 11, 2010 at 3:06 PM, Nick Coghlan wrote: > Basically, as of 3.2, the correct "default" use is likely to be: > > print() for information you want to appear on stdout by default > (especially in scripts) > logging.debug() and .info() for additional debugging information that > should be silenced by default > > warnings.warn() for actual programmatic warnings that should obey > warning filter rules (logging can hook the warnings display mechanism) > logging.warn() for warnings that should never raise an exception and > should be emitted every time they are encountered > > raise an exception for normal error reporting > logging.error(), .critical() and .exception() for error reporting that > cannot raise an exception for some reason (e.g. long running processes > that are simply recording that an exception occurred) This could actually make a reasonably good basic for a "task oriented" subsection of the logging documentation. Something like: === Task: Display console output for ordinary usage of a command line script or program Tool: Use print() Task: Report events that occur during normal operation of a program (e.g. for status monitoring or fault investigation) Tool: Use logging.info() (or logging.debug() for especially voluminous or expensive to calculate output) Task: Issue a warning regarding a particular runtime event Tool: Use warnings.warn() in library code if the issue is avoidable and the client application should be modified to eliminate the warning. Use logging.warn() if there is nothing the client application can do about the situation, but the event should still be noted Task: Report an error regarding a particular runtime event Tool: Raise an exception Task: Report suppression of an error without raising an exception (e.g. error handler in a long-running server process) Tool: Use logging.error(), logging.exception(), or logging.critical() as appropriate to the specific error and application domain Note: all references to the root level logging functions in the above recommendations also cover direct invocation of the corresponding methods on specific logger instances. === Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r87162 - python/branches/release27-maint/Doc/library/stdtypes.rst
On Sat, Dec 11, 2010 at 10:41 AM, raymond.hettinger wrote: > Author: raymond.hettinger > Date: Sat Dec 11 01:41:02 2010 > New Revision: 87162 > > Log: > Issue 2690: Doc fixup. xrange() objects are slicable. > > > Modified: > python/branches/release27-maint/Doc/library/stdtypes.rst Are you sure about that one? Python 2.7.0+ (trunk:86033, Nov 1 2010, 00:42:57) [GCC 4.4.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> xrange(10)[1:2] Traceback (most recent call last): File "", line 1, in TypeError: sequence index must be integer, not 'slice' Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Nick, Thanks for the detailed exposition. > Notice that even the recommended "basicConfig" approach to resolving > this is subtly flawed, since your choices are to either display info Right. basicConfig() is pretty basic. > Basically, as of 3.2, the correct "default" use is likely to be: The suggestions make sense. > [1] As an exercise, I decided to actually configure recommended > console logging output style via the logging module. It proved to be > quite informative as to where some of the flaws in the current > documentation lie. [snip] > documentation of Logger.setLevel()! There really should be a section > early on in the logging documentation describing the root logger, how Agreed. > In addition to that, the documentation of the LogRecord class should > really include a table of attribute names that are available for use > in formatting and filtering. Initially I thought the attribute names Yes. > formatting (when the documentation for the new styles feature goes in > would probably be an appropriate time to fix this). Similarly, the Sorry, what do you mean by "new styles feature"? > Anyway, the shortest way I could find of setting this up (debug Yes, except that you could have set the root logger level to INFO rather than DEBUG. This raises some more questions. Obviously, we don't want users to have to go through these steps themselves. Once these steps have been finalised, the last resort handler could be modified to do all this. Should this handler be part of the public API, so that it can be inherited from etc? I realise we're in beta and hence feature freeze ... just sayin'. We're all consenting adults here, after all :-) Regards, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
Nick Coghlan gmail.com> writes: > This could actually make a reasonably good basic for a "task oriented" > subsection of the logging documentation. Something like: > Good suggestion, I'll see what I can do. Thanks, Vinay Sajip ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Using logging in the stdlib and its unit tests
On 12/10/2010 9:06 PM, Nick Coghlan wrote: Anyway, the shortest way I could find of setting this up (debug silenced, info written to stdout, warning and above written to stderr): import sys, logging root = logging.getLogger() # Turns out the level of the root logger is set to WARNING by default # That should be much easier to find out from the logging docs! root.setLevel(logging.DEBUG) # Send WARNING and above to stderr # basicConfig() is no help, since it sets the specified level on the root logger # but NOT on the default handler it installs. To emulate console output # we want to skip the formatting anyway. err = logging.StreamHandler() err.setLevel(logging.WARNING) root.addHandler(err) # Send INFO to stdout def is_info(record): return record.levelno == logging.INFO out = logging.StreamHandler(stream=sys.stdout) out.addFilter(is_info) root.addHandler(out) Wow, that's way longer than I expected... almost as long as my "cheap logger". Greg didn't flesh out the "setup" stuff, but you have, so this is educational. Thanks. Not sure I want my INFO to go to stdout, by the way. That corresponds to "debugging print statement" behavior, but even my "cheap logger" actually puts stuff in a file instead, and gets it out of the way of the normal output. I'd suggest that this be "simplified" to put all the messages in the same (disk) file, but then I'm sure that there are other preferences than mine... you clearly thought about what you wanted/thought would be useful, in coming up with the above. On 12/10/2010 9:24 PM, Nick Coghlan wrote: This could actually make a reasonably good basic for a "task oriented" subsection of the logging documentation. Something like: Yep, agree. But sadly, for each point, there may be multiple options (your StreamHandler, but I'd want a FileHandler; your separation of messages by level, my wanting them combined; etc.) Your comment about basicConfig setting the level on the root logger, but not on the default handler making it useless is opaque to me, but is there perhaps room for another basic setup API that could get the setup code down to a line or two in simple cases? Maybe 3 parameters: 1. minimum level to be generated, which would be passed through to the root logger and anything else defined by this basic setup API 2. whether debug and info should go to the same or different stream/file as warn+ messages. Maybe this is a stream or filename or None, the last implying to use the warn+ output. 3. Where to send the warn+ output. maybe a 4. Maximum (size, count) of the round-robin log files, assuming that either or both of 2 & 3 specify files. Would that be a useful set of functionality to bundle? And could it be extended, when the user wants more power, or would it have to be replaced, because it gets in the way of the user that wants more power? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
