Logging module: problem with some mapping keys

[Tekkaman]

I'm getting a strange behaviour from the "pathname" and "lineno"
formatter mapping keys. Instead of my file and my line number I get:

/usr/lib/python2.4/logging/__init__.py

as the file, and 1072 as the line number. I set up my config as
follows:

logBaseConf = {
'level'   : logging.DEBUG,
'format'  : "%(levelname)-8s|%(asctime)s|%(pathname)s,
%(name)s, line %(lineno)s|%(message)s",
'datefmt ': '%d %b %Y, %H:%M:%S',
'filename': 'logtest.log',
'filemode': 'a'
}
logging.basicConfig(**logBaseConf)

I'm not using any executable-generating tools such as cx_Freeze or
Psyco. In fact, I'm getting this error on a very basic script with the
specific purpose of testing the logging module capabilities.

Thanks in advance for any contribution.

T.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Logging module: problem with some mapping keys

[Tekkaman]

sim.sim wrote:
> Hi, i've check documentation, and found that logging.basicConfig takes
> no arguments (probably we have different versions of logging package),

Your Python version is < 2.4. Now basicConfig takes optional keyword
args:

basicConfig([**kwargs])
Does basic configuration for the logging system by creating a
StreamHandler with a default Formatter and adding it to the root
logger. The functions debug(), info(), warning(), error() and
critical() will call basicConfig() automatically if no handlers are
defined for the root logger.

In my real application (not the small test script!) This behaviour is
very useful since I can call basicConfig only once in the main file and
then all I have to do in the other files is:

import logging
self.logger = logging.getLogger("myClass")

Everything works smoothly except for not being able to get file and
line number info.

> and i've never used it.
>
> just try this:
>
>
> fileName = 'testlog.log'
> logName = 'LOG'
> iHandler = logging.FileHandler(fileName)
> iHandler.setFormatter(
> logging.Formatter("%(levelname)-8s|%(asctime)s|%(pathname)s,%(name)s,
> line %(lineno)s|%(message)s") )
>
> iLog = logging.getLogger(logName)
> iLog.addHandler(iHandler)
> iLog.setLevel(logging.DEBUG)
>
> iLog.info('hello logging')
>
>
>
> it gives me:
>
> INFO|2006-12-13 19:57:33,575|test.py,LOG, line 12|hello logging
>
>
> On 13 Dec., 19:02, "Tekkaman" <[EMAIL PROTECTED]> wrote:
> > I'm getting a strange behaviour from the "pathname" and "lineno"
> > formatter mapping keys. Instead of my file and my line number I get:
> >
> > /usr/lib/python2.4/logging/__init__.py
> >
> > as the file, and 1072 as the line number. I set up my config as
> > follows:
> >
> > logBaseConf = {
> > 'level'   : logging.DEBUG,
> > 'format'  : "%(levelname)-8s|%(asctime)s|%(pathname)s,
> > %(name)s, line %(lineno)s|%(message)s",
> > 'datefmt ': '%d %b %Y, %H:%M:%S',
> > 'filename': 'logtest.log',
> > 'filemode': 'a'
> > }
> > logging.basicConfig(**logBaseConf)
> >
> > I'm not using any executable-generating tools such as cx_Freeze or
> > Psyco. In fact, I'm getting this error on a very basic script with the
> > specific purpose of testing the logging module capabilities.
> >
> > Thanks in advance for any contribution.
> > 
> > T.
> 
> --
> Maksim Kasimov

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Logging module: problem with some mapping keys

[Tekkaman]

Peter Otten wrote:
>
> An evil symlink, maybe?
Thanks for the hint, but it's not a symlink
>
> $ ln -s /usr/local/lib/python2.4/logging
> $ python
> [snip]
> >>> import logging
> >>> logging.basicConfig(format="%(pathname)s")
> >>> logging.getLogger().critical("yadda")
> /usr/local/lib/python2.4/logging/__init__.py
> >>>
> $ rm logging
> $ python
> [snip]
> >>> import logging
> >>> logging.basicConfig(format="%(pathname)s")
> >>> logging.getLogger().critical("yadda")
> 
> 
> Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Logging module: problem with some mapping keys

[Tekkaman]

'/usr/lib64/python2.4/logging/__init__.py'

Peter Otten wrote:
> Tekkaman wrote:
>
> > Thanks for the hint, but it's not a symlink
>
> What does logging._srcfile contain? Should be
>
> >>> import logging
> >>> logging._srcfile
> '/usr/lib/python2.4/logging/__init__.py'
> 
> on your machine, but probably isn't.
> 
> Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Logging module: problem with some mapping keys

[Tekkaman]

lib is a symlink to lib64

Peter Otten wrote:
> Tekkaman wrote:
>
> > Peter Otten wrote:
> >> Tekkaman wrote:
> >>
> >> > Thanks for the hint, but it's not a symlink
> >>
> >> What does logging._srcfile contain? Should be
> >>
> >> >>> import logging
> >> >>> logging._srcfile
> >> '/usr/lib/python2.4/logging/__init__.py'
> >>
> >> on your machine, but probably isn't.
>
> > '/usr/lib64/python2.4/logging/__init__.py'
>
> And neither /usr/lib nor /usr/lib/python2.4 are symlinks?
> 
> Peter
> 
> PS: Please don't top-post

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Logging module: problem with some mapping keys

[Tekkaman]

Peter Otten wrote:
> Tekkaman wrote:
>
> > Peter Otten wrote:
> >> Tekkaman wrote:
> >>
> >> > Thanks for the hint, but it's not a symlink
> >>
> >> What does logging._srcfile contain? Should be
> >>
> >> >>> import logging
> >> >>> logging._srcfile
> >> '/usr/lib/python2.4/logging/__init__.py'
> >>
> >> on your machine, but probably isn't.
>
> > '/usr/lib64/python2.4/logging/__init__.py'
>
> And neither /usr/lib nor /usr/lib/python2.4 are symlinks?
>
> Peter
>
> PS: Please don't top-post

Sry for the top-post, I noticed the PS a moment too late!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Logging module: problem with some mapping keys

[Tekkaman]

On Dec 14, 6:41 pm, Peter Otten <[EMAIL PROTECTED]> wrote:
> Tekkaman wrote:
>> lib is a symlink to lib64
> So my initial diagnosis was correct. Unfortunately I can no longer recommend
> that you remove the symlink...
>
Yes. What I really meant in my first reply was "it's not a symlink in
the script's own directory": I didn't suspect that a symlink at any
level in the path between the script and the logging module would break
findCaller().
> Putting /usr/lib64/python2.4 as the first entry into your
>
> PYTHONPATH
>
> environment variable might fix the problem (the idea is
> that /usr/lib64/python2.4 precedes /usr/lib/python2.4 in sys.path and is
> therefore used for the import of the logging package).
>
Thanks, I'll check it out. Anyway, since this is a hack, you think this
behaviour should be reported as a bug?
> Peter
T.

-- 
http://mail.python.org/mailman/listinfo/python-list

unique elements from list of lists

[Tekkaman]

I have a list of lists and I want to define an iterator (let's call
that uniter) over all unique elements, in any order. For example,
calling:

sorted(uniter([['a', 'b', 'd'], ['b', 'c'], ['a', 'c', 'd']]))

must return ['a', 'b', 'c', 'd']. I tried the following
implementations:

from itertools import chain
def uniter1(listOfLists):
for item in set(chain(*listOfLists)): yield item

def uniter2(listOfLists):
for item in reduce(
lambda x,y: x|y,
[set(list_) for list_ in listOfLists]
): yield item

speed test with timeit says the first one is slightly faster. What
bothers me is that it builds a set from an iterator and then another
iterator from the set. Is there a way to implement this using only
iterators? I also tried a "full python" implementation (by that I mean
one that does not use the built-in set and keeps track of previously
yielded items in a list) but the one I pulled out is about 180 times
slower. Here it is:

def uniter3(listOfLists):
done = []
for list_ in listOfLists:
for item in list_:
if not item in done:
done.append(item)
yield item

Thanks in advance for any contribution.

-- Simone

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: unique elements from list of lists

[Tekkaman]

Thanks everybody!
Azrael: your suggestions involve python-level membership testing and
dummy list construction just like my uniter3 example, so I'm afraid
they would not go any faster. There's no built-in sequence flattening
function that I know of btw.
bearophile: your implementation is very similar to my uniter2
(subsequent set unions) but without the additional lambda def
overhead, and in fact it goes faster. Not faster than uniter though.
Peter: your solution is the fastest, removing the explicit for loop
resulted in a 30% speed gain. Looks like using set() is a must.

-- Simone

-- 
http://mail.python.org/mailman/listinfo/python-list