Re: xlrd 0.8.0 released!

[pruebauno]

On Wednesday, August 1, 2012 11:01:56 AM UTC-4, Chris Withers wrote:
> Hi All,
> 
> 
> 
> I'm pleased to announce the release of xlrd 0.8.0:
> 
> 
> 
> http://pypi.python.org/pypi/xlrd/0.8.0
> 
> 
> 
> This release finally lands the support for both .xls and .xlsx files.
> 
> Many thanks to John Machin for all his work on making this happen.
> 
> Opening of .xlsx files is seamless, just use xlrd as you did before and 
> 
> it all should "just work".
> 
> 
> 
> xlrd 0.8.0 is also the first release that that targets Python 2.6 and 
> 
> 2.7, but no Python 3 just yet. Python 2.5 and below may work but are not 
> 
> supported. If you need to use Python 2.5 or earlier, please stick to 
> 
> xlrd 0.7.x.
> 
> 
> 
> Speaking of xlrd 0.7.x, that's now in "requested maintenance only" mode 
> 
> ;-) That means, if possible, use 0.8.x. If you have a really good reason 
> 
> for sticking with 0.7.x, and you find a bug that you can't work around, 
> 
> then please make this clear on the python-ex...@googlegroups.com and 
> 
> we'll see what we can do.
> 
> 
> 
> If you find any problems, please ask about them on the list, or submit 
> 
> an issue on GitHub:
> 
> 
> 
> https://github.com/python-excel/xlrd/issues
> 
> 
> 
> Full details of all things Python and Excel related can be found here:
> 
> 
> 
> http://www.python-excel.org/
> 
> 
> 
> cheers,
> 
> 
> 
> Chris
> 
> 
> 
> -- 
> 
> Simplistix - Content Management, Batch Processing & Python Consulting
> 
>  - http://www.simplistix.co.uk

Congrats! Being able to read Office 2007 files will be very useful. Looking 
forward to the Python 3 support.
-- 
http://mail.python.org/mailman/listinfo/python-list

tricky time interval billing problem

[pruebauno]

I am currently working on a tricky problem at work. I googled around a
bit, but "time intervals" did not come up with anything useful.
Although I have some rough idea of how I could solve it, I still would
value some input.

I have information of (It has only couple dozen entries.)
ServiceNum, DollarCost

and a input database table in the form (several GBytes):
ClientNumber (CN), BeginDate(BD), EndDate(ED),
ServiceNumber_Purchased(SN)
--Date intervals are always [closed, closed]

The output is:
ClientNumber(CN), BeginDate(BD), Enddate(ED), DollarCost(DC)

With the following requirements:
1) The input dates can be overlapping dates.
The output has to be non-overlapping "broken up" dates

Example: (assuming SN 1=$10 and SN 2=$20)
input (CN, BD, ED ,SN):
10, 1/1/2001,1/1/2005,1
10, 1/1/2002,1/1/2004,2

should result in:
output (CN, BD, ED, DC):
10, 1/1/2001, 12/31/2001, $10
10, 1/1/2002, 1/1/2004, $30
10, 1/2,2004, 1/1/2005, $10

2) if the same service is purchased twice for an interval it should be
billed only once
Example:
input:
10, 1/1/2001, 1/1/2005, 1
10, 1/1/2004, 1/1/2007, 1

output:
10, 1/1/2001, 1/1/2007, $10

Here are my thoughts about a solution so far:

1. fetch the input data sorted by clientNumber, Begindate, Enddate and
ServiceNumber

2. read the row, store as temporary good interval, then read another
row

3. if new begin-date is bigger than previous good interval end-date (or
previously saved end-date), output previous good interval, start new
good interval

4. else create new good interval entry with good interval begin-date
and current row begin-date, store good interval end-date into a list
with bisect module or something (so we always compare against smallest
end-date).

5. Use "bitwise or" on a service bitfield to add services and caluate
the total

As you can see my algorithm is a bit scetchy right now, but at this
point I am more interested to determine if the bisect module would be
the best way to approach the date interval problem or if I should use
some other data structure instead (stack, queue, set,...). I am working
on a Python proof of concept now. It is likely that the company will
want a C++ version of it later (yikes).

nes

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: tricky time interval billing problem

[pruebauno]

> First of all, you need to use ordering to ensure that the database gives
> you the most convenient order for processing, as this will make your
> computations much easier. So I'd suggest sorting by clientNumber,
> ServiceNumber, Begindate and Enddate. That way you can consider each
> service separately.
>
> Then all you need to do is accumulate the entries where the clientNumber
> and ServiceNumber are the same and the start date of the second is less
> than or equal to the end date of the first, repeatedly until either
> there's no date overlap or a new service and/or client is started.
>
> This shouldn't need any intermediate storage of results: if the row
> you've just read can be used to extend the current range then extend it,
> otherwise emit the current range and replace it with the new row.
>
> Or is there something I've failed to understand about how you want to
> process the data?
>
> regards
>   Steve

Hi Steve,  (btw I am the one that sent you the quotes at Pycon)

Hm, that would be an algorithm for requirement number 2. I do not
understand how it would help with respect to requirement 1. Notice that
by coincidence, in my examples the input data is already sorted as you
specified.

The real data is of course more messy than my analogy and I will post
it here so that nobody can accuse me of "feature creep". I hope I don't
totally confuse you now. Feel free to ignore it. The real output is not
100% correct either (that is why I am rewriting the program). Some of
the database in all its gory, ..eh glory:

INPUT
43756352|D|01/01/1999|09/30/2003|DCUD2B00|DCUD2B00|Y|A|43756350|D|83516
  |00374|9048327561|0001|
43756353|D|01/01/1999|09/30/2003|DCUD2B00|DCUD2B00|Y|A|43756350|D|83516
  |00374|9048327561|0001|
43756351|D|01/01/1999|09/02/2002|DCUD2B00|DCUD2B00|Y|A|43756350|M|83516
  |00374|9048327561|0001|
43756354|D|04/02/1999|09/30/2003|DCUD2B00|DCUD2B00|Y|A|43756350|W|83516
  |00374|9048327561|0001|
43756351|M|01/01/1999|03/31/1999|MARTPPG2|MARTPPG2|Y|A|43756350|M|83516
  |00374|9048327561|0001|
43756352|M|01/01/1999|03/31/1999|MARTPPG2|MARTPPG2|Y|A|43756350|D|83516
  |00374|9048327561|0001|
43756353|M|01/01/1999|03/31/1999|MARTPPG2|MARTPPG2|Y|A|43756350|D|83516
  |00374|9048327561|0001|
43756351|M|04/01/1999|07/31/2002|MBCPG002|MBCPG002|Y|A|43756350|M|83516
  |00374|9048327561|0001|
43756352|M|04/01/1999|07/31/2002|MBCPG002|MBCPG002|Y|A|43756350|D|83516
  |00374|9048327561|0001|
43756353|M|04/01/1999|07/31/2002|MBCPG002|MBCPG002|Y|A|43756350|D|83516
  |00374|9048327561|0001|
43756354|M|04/02/1999|07/31/2002|MBCPG002|MBCPG002|Y|A|43756350|W|83516
  |00374|9048327561|0001|
43756352|M|08/01/2002|09/30/2003|MBP07305|MBP07305|Y|A|43756350|D|83516
  |00374|9048327561|0001|
43756353|M|08/01/2002|09/30/2003|MBP07305|MBP07305|Y|A|43756350|D|83516
  |00374|9048327561|0001|
43756354|M|08/01/2002|09/30/2003|MBP07305|MBP07305|Y|A|43756350|W|83516
  |00374|9048327561|0001|
43756351|M|08/01/2002|09/02/2002|MBP07305|MBP07305|Y|A|43756350|M|83516
  |00374|9048327561|0001|

OUTPUT
43756350|9048327561|DCUD2B00|D|A|01/01/1999|04/01/1999|83516   |00374
 |0001|Y|A|DCUD2B00|
43756350|9048327561|DCUD2B00|D|E|04/02/1999|09/30/2003|83516   |00374
 |0001|Y|A|DCUD2B00|
43756350|9048327561|MARTPPG2|M|A|01/01/1999|03/31/1999|83516   |00374
 |0001|Y|A|MARTPPG2|
43756350|9048327561|MBCPG002|M|A|04/01/1999|07/31/2002|83516   |00374
 |0001|Y|A|MBCPG002|
43756350|9048327561|MBP07305|M|A|08/01/2002|09/02/2002|83516   |00374
 |0001|Y|A|MBP07305|
43756350|9048327561|MBP07305|M|E|09/03/2002|09/30/2003|83516   |00374
 |0001|Y|A|MBP07305|

CHEAT SHEET:
|   (M)  |  (H,W)  |  (D,O,S)  ||
+==
|   -|-| - ||O|
+-+
|   -|-| X ||G|
+-+
|   -|X| - ||F|
+-+
|   -|X| X ||E|
+-+
|   X|-| - ||C|
+-+
|   X|-| X ||D|
+-+
|   X|X| - ||B|
+-+
|   X|X| X ||A|
+-+

regards,
Nes

-- 
http://mail.python.org/mailman/listinfo/python-list

Issues compiling with large file support

[pruebauno]

Hello all,
I am having issues compiling Python with large file support. I tried
forcing the configure script to add it but then it bombs in the make
process. Any help will be appreciated.

Information:
Architecture:
PowerPc on AIX version 5

Compiler:
VisualAge C++ Professional / C for AIX Compiler, Version 6

Python version:
Python 2.4.1

Error Message:
cc_r -c  -DNDEBUG -g -O2 -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64
-I
. -I./Include  -DPy_BUILD_CORE -o Objects/fileobject.o
Objects/fileobject.c
"Objects/fileobject.c", line 382.2: 1506-205 (S) #error "Large file
support, but neither off_t nor fpos_t is large enough."
"Objects/fileobject.c", line 389.27: 1506-046 (S) Syntax error.
"Objects/fileobject.c", line 423.2: 1506-205 (S) #error "Large file
support, but no way to fseek."
"Objects/fileobject.c", line 431.8: 1506-166 (S) Definition of function
Py_off_t requires parentheses.
"Objects/fileobject.c", line 432.1: 1506-276 (S) Syntax error: possible
missing
'{'?"Objects/fileobject.c", line 446.2: 1506-205 (S) #error "Large file
support, but no way to ftell."
"Objects/fileobject.c", line 456.18: 1506-275 (S) Unexpected text
offset encountered.
"Objects/fileobject.c", line 456.9: 1506-045 (S) Undeclared identifier
Py_off_t.
"Objects/fileobject.c", line 468.9: 1506-045 (S) Undeclared identifier
offset.
"Objects/fileobject.c", line 476.48: 1506-099 (S) Unexpected argument.
"Objects/fileobject.c", line 494.18: 1506-275 (S) Unexpected text
newsize encountered.
"Objects/fileobject.c", line 494.9: 1506-045 (S) Undeclared identifier
Py_off_t.
"Objects/fileobject.c", line 496.18: 1506-275 (S) Unexpected text
initialpos encountered.
"Objects/fileobject.c", line 513.9: 1506-045 (S) Undeclared identifier
initialpos.
"Objects/fileobject.c", line 525.17: 1506-045 (S) Undeclared identifier
newsize.
"Objects/fileobject.c", line 586.52: 1506-099 (S) Unexpected argument.
"Objects/fileobject.c", line 604.18: 1506-275 (S) Unexpected text pos
encountered.
"Objects/fileobject.c", line 604.9: 1506-045 (S) Undeclared identifier
Py_off_t.
"Objects/fileobject.c", line 610.9: 1506-045 (S) Undeclared identifier
pos.
make: 1254-004 The error code from the last command is 1.

Stop.


More info:

checking MACHDEP... aix5
checking EXTRAPLATDIR...
checking for --without-gcc...
checking for --with-cxx=... no
checking for c++... no
checking for g++... no
checking for gcc... no
checking for CC... no
checking for cxx... no
checking for cc++... no
checking for cl... no
checking for gcc... cc_r
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... no
checking whether cc_r accepts -g... yes
checking for cc_r option to accept ANSI C... none needed
checking how to run the C preprocessor... cc_r -E
checking for egrep... grep -E
checking for AIX... yes
checking for --with-suffix...
checking for case-insensitive build directory... no
checking LIBRARY... libpython$(VERSION).a
checking LINKCC... $(srcdir)/Modules/makexp_aix Modules/python.exp .
$(LIBRARY);
 $(PURIFY) $(CC)
checking for --enable-shared... no
checking for --enable-profiling...
checking LDLIBRARY... libpython$(VERSION).a
checking for ranlib... ranlib
checking for ar... ar
checking for a BSD-compatible install... ./install-sh -c
checking for --with-pydebug... no
checking whether cc_r accepts -OPT:Olimit=0... no
checking whether cc_r accepts -Olimit 1500... no
checking whether pthreads are available without options... yes
checking for ANSI C header files... yes

-- 
http://mail.python.org/mailman/listinfo/python-list

Improving my text processing script

[pruebauno]

I am sure there is a better way of writing this, but how?

import re
f=file('tlst')
tlst=f.read().split('\n')
f.close()
f=file('plst')
sep=re.compile('Identifier "(.*?)"')
plst=[]
for elem in f.read().split('Identifier'):
content='Identifier'+elem
match=sep.search(content)
if match:
plst.append((match.group(1),content))
f.close()
flst=[]
for table in tlst:
for prog,content in plst:
if content.find(table)>0:
flst.append('"%s","%s"'%(prog,table))
flst.sort()
for elem in flst:
print elem



What would be the best way of writing this program. BTW find>0 to check
in case table=='' (empty line) so I do not include everything.

tlst is of the form:

tablename1
tablename2

...

plst is of the form:

Identifier "Program1"
Name "Random Stuff"
Value "tablename2"
...other random properties
Name "More Random Stuff"
Identifier "Program 2"
Name "Yet more stuff"
Value "tablename2"
...


I want to know in what programs are the tables in tlst (and only those)
used.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Improving my text processing script

[pruebauno]

Paul McGuire wrote:
> match...), this program has quite a few holes.
>
> What if the word "Identifier" is inside one of the quoted strings?
> What if the actual value is "tablename10"?  This will match your
> "tablename1" string search, but it is certainly not what you want.
> Did you know there are trailing blanks on your table names, which could
> prevent any program name from matching?

Good point. I did not think about that. I got lucky because none of the
table names had trailing blanks (google groups seems to add those) the
word identifier is not used inside of quoted strings anywhere and  I do
not have tablename10, but I do have "dba.tablename1" and that one has
to match with tablename1 (and magically did).

>
> So here is an alternative approach using, as many have probably
> predicted by now if they've spent any time on this list, the pyparsing
> module.  You may ask, "isn't a parser overkill for this problem?" and

You had to plug pyparsing! :-). Thanks for the info I did not know
something like pyparsing existed. Thanks for the code too, because
looking at the module it was not totally obvious to me how to use it. I
tried run it though and it is not working for me. The following code
runs but prints nothing at all:

import pyparsing as prs

f=file('tlst'); tlst=[ln.strip() for ln in f if ln]; f.close()
f=file('plst'); plst=f.read()  ; f.close()

prs.quotedString.setParseAction(prs.removeQuotes)

identLine=(prs.LineStart()
  + 'Identifier'
  + prs.quotedString
  + prs.LineEnd()
  ).setResultsName('prog')

tableLine=(prs.LineStart()
  + 'Value'
  + prs.quotedString
  + prs.LineEnd()
  ).setResultsName('table')

interestingLines=(identLine | tableLine)

for toks,start,end in interestingLines.scanString(plst):
print toks,start,end

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Improving my text processing script

[pruebauno]

Miki Tebeka wrote:

> Look at re.findall, I think it'll be easier.

Minor changes aside the interesting thing, as you pointed out, would be
using re.findall. I could not figure out how to.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Improving my text processing script

[pruebauno]

[EMAIL PROTECTED] wrote:
> Paul McGuire wrote:
> > match...), this program has quite a few holes.

> tried run it though and it is not working for me. The following code
> runs but prints nothing at all:
>
> import pyparsing as prs
>
And this is the point where I have to post the real stuff because your
code works with the example i posted and not with the real thing. The
identifier I am interested in is (if I understood the the requirements
correctly) the one after the "title with the stars"

So here is the "real" data for tlst some info replaced with z to
protect privacy:

*


   Identifier "zzz0main"


*


   Identifier "zz501"


 Value "zzz_CLCL_,zz_ID"


 Name "z"


 Name "zz"


*


   Identifier "3main"


*


   Identifier "zzz505"


 Value "dba.zzz_CKPY__SUM"


 Name "xxx_xxx_xxx_DT"


--


 Value "zzz__zzz_zzz"


 Name "zzz_zz_zzz"


--


 Value "zzz_zzz_zzz_HIST"


 Name "zzz_zzz"


--


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: anaconda.real in RH7.1

[pruebauno]

Allan Adler wrote:
> I'm using the book, "Programming Python", by Mark Lutz, as a reference.

No offence to Mark Lutz or O'Reilly but I consider "Programming Python"
one of the worst books I have read (in my case an old first edition).
It overwhelms the beginning programmer ("Learning Python" is probably
better for that), it bores the experienced programmer to death with
introductory details and does not work as a reference book. It is a
nice survey of application areas for python, but the book lacks a clear
goal, purpose and a problem it tries to address. It needs more focus.
It needs to be marketed correctly also. I believe too many people new
to python buy that book and get discouraged about Python, O'Reilly and
programming when they should buy something else instead.

That said what you probably want is "Python in a Nutshell" by O'Reilly
which is a good reference book it has a concise introduction of Python
in the beginning and after that, documentation for the most usefull
libraries out there.

Personally I use the online documentation a lot:

If I know the the name of the module
(for example it starts with sys.zz) I use:
http://www.python.org/doc/2.4.1/modindex.html

If I know what I want but not the name I use:
http://www.python.org/doc/2.4.1/lib/lib.html

-- 
http://mail.python.org/mailman/listinfo/python-list

compiling on AIX 5.3 with vacpp

[pruebauno]

I am trying to compile Python 2.5 on AIX 5.3. I used

export PATH=/usr/bin:/usr/vacpp/bin

OPT=-O2 ./configure --with-gcc="xlc_r -q64" --with-cxx="xlC_r -q64"
--disable-ipv6 AR="ar -X64"

make

The following error stops make in its track:

building '_locale' extension
./Modules/ld_so_aix xlc_r -q64 -bI:Modules/python.exp
build/temp.aix-5.3-2.5/home/pxadm/.test/Python-2.5/Modules/_localemodule.o
-L/usr/local/lib -o build/lib.aix-5.3-2.5/_locale.so
ld: 0711-317 ERROR: Undefined symbol: .bindtextdomain
ld: 0711-317 ERROR: Undefined symbol: .textdomain
ld: 0711-317 ERROR: Undefined symbol: .dcgettext
ld: 0711-317 ERROR: Undefined symbol: .dgettext
ld: 0711-317 ERROR: Undefined symbol: .gettext
ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
information.
*** WARNING: renaming "_locale" since importing it failed:
0509-022 Cannot load module build/lib.aix-5.3-2.5.
0509-026 System error: A file or directory in the path name
does not exist.
error: No such file or directory
make: 1254-004 The error code from the last command is 1.

Stop.

>

Any help appreciated as to explain what that error could mean and
possible ways of fixing it.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: compiling on AIX 5.3 with vacpp

[pruebauno]

[EMAIL PROTECTED] wrote:
> [EMAIL PROTECTED] wrote:
> > I am trying to compile Python 2.5 on AIX 5.3. I used
> >
> > building '_locale' extension
> > ./Modules/ld_so_aix xlc_r -q64 -bI:Modules/python.exp
> > build/temp.aix-5.3-2.5/home/pxadm/.test/Python-2.5/Modules/_localemodule.o
> > -L/usr/local/lib -o build/lib.aix-5.3-2.5/_locale.so
> > ld: 0711-317 ERROR: Undefined symbol: .bindtextdomain
> > ld: 0711-317 ERROR: Undefined symbol: .textdomain
> > ld: 0711-317 ERROR: Undefined symbol: .dcgettext
> > ld: 0711-317 ERROR: Undefined symbol: .dgettext
> > ld: 0711-317 ERROR: Undefined symbol: .gettext
>
> The problem is that an additional library is needed to link the locale
> module.  You will first need to determine what library is needed.
> Maybe libintl?  To find out, you can do a man on any of those symbols,
> for example, man textdomain.
>
> That should tell you want library is required.  You will then need to
> modify setup.py to add the extra library for the locale module.
> There's already some support for this in setup.py around line 390:
>
> # access to ISO C locale support
> data = open('pyconfig.h').read()
> m = re.search(r"#s*define\s+WITH_LIBINTL\s+1\s*", data)
> if m is not None:
> locale_libs = ['intl']
> else:
> locale_libs = []
> if platform == 'darwin':
> locale_extra_link_args = ['-framework', 'CoreFoundation']
> else:
> locale_extra_link_args = []
>
>
> exts.append( Extension('_locale', ['_localemodule.c'],
>libraries=locale_libs,
>extra_link_args=locale_extra_link_args)
> )
>
> Once you get something working, please post a patch.
>
> n

Thanks for the info Neil,

sorry for taking long to answer. This is skunkwork, I am working on it
whenever I get some time.

indeed as you say. libintl.h is required so I hard coded it into
setup.py to no effect. At what point does setup.py get called? by make?

my code
# access to ISO C locale support
data = open('pyconfig.h').read()
m = re.search(r"#s*define\s+WITH_LIBINTL\s+1\s*", data)
if m is not None:
locale_libs = ['intl']
else:
locale_libs = []
if platform == 'darwin':
locale_extra_link_args = ['-framework', 'CoreFoundation']
else:
locale_extra_link_args = []

locale_libs=['intl']

exts.append( Extension('_locale', ['_localemodule.c'],
   libraries=locale_libs,
   extra_link_args=locale_extra_link_args) )

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Compiling python2.5 on IBM AIX

[pruebauno]

On Jul 17, 3:58 am, [EMAIL PROTECTED] wrote:
> > I haven't compiled it myself, but I'm told that the installation I
> > work with was compiled with:
>
> > export PATH=$PATH:/usr/vacpp/bin:/usr/vacpp/lib
> > ./configure --with-gcc="xlc_r -q64" --with-cxx="xlC_r -q64" --disable-
> > ipv6 AR="ar -X64"
> > make
> > make install
>
> I've tried with the followong configuration :
> --
> export PATH=$PATH:/usr/vacpp/bin:/usr/vacpp/lib
> ./configure --prefix=${BASE} --with-gcc="xlc_r -q64" --with-cxx="xlC_r
> -q64" --disable-ipv6 AR="ar -X64" LDFLAGS="-L\${BASE}/lib/" PPFLAGS="-I
> \${BASE}/include/"
>
> make
> -
>
> but it doesn't compile either :
>
> make
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Modules/python.o ./Modules/python.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Modules/_typesmodule.o Modules/_typesmodule.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/acceler.o Parser/acceler.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/grammar1.o Parser/grammar1.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/listnode.o Parser/listnode.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/node.o Parser/node.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/parser.o Parser/parser.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/parsetok.o Parser/parsetok.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/bitset.o Parser/bitset.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/metagrammar.o Parser/metagrammar.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/firstsets.o Parser/firstsets.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/grammar.o Parser/grammar.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/pgen.o Parser/pgen.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/myreadline.o Parser/myreadline.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Parser/tokenizer.o Parser/tokenizer.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Objects/abstract.o Objects/abstract.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Objects/boolobject.o Objects/boolobject.c
> xlc_r -q64 -c  -DNDEBUG -O  -I. -I./Include   -DPy_BUILD_CORE -
> o Objects/bufferobject.o Objects/bufferobject.c
> "Objects/bufferobject.c", line 22.15: 1506-275 (S) Unexpected text ','
> encountered.
> make: 1254-004 The error code from the last command is 1.
>
> any idea ?
>
> thanks
>
> --
> BL

Remove the extra comma at the end of line 22 of bufferobject.c
That should help you fix it.

I sent a bug report for that:

http://sourceforge.net/tracker/index.php?func=detail&aid=1557983&group_id=5470&atid=105470


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Dict Help

[pruebauno]

On Jul 17, 5:57 am, "Robert Rawlins - Think Blue"
<[EMAIL PROTECTED]> wrote:
> Morning Gabriel,
>
> I'm looking for a little more advice on this dictionary/list to
> defaultdict/set conversion that we were talking about, there were a few
> things I was looking to clarify. Firstly, what is the difference between a
> standard dict and a default dict? Is it purely a performance issue?
>
> This dict is likely to grow pretty large and is read/written on a very
> regular basis so the better performing one is going to work best for me.
> Also, am I still able to iterate over a set in the same way I can a list?
>
> Here is an example of my add function at the moment, how can that be
> converted to a defaultdict/set instead of the dict/list approach?
>
> self.History = {}
>
> def addHistory(self, address, media):
> if address not in self.History:
> self.History[address] = []
>
> self.History[address].append(media)
>
> Thanks Gabriel,
>
> Rob
>
1. defaultdict just sets a dictionary entry to a default if the key
does not exist. In other words if you replace the first line with:

self.History=defaultdict(list) #replace list by set if you want

you can get rid of:

if address not in self.History: self.History[address]=[]

because that then happens automatically. Otherwise it works exactly
like a regular dictionary.

2. You can iterate over a set the same way you iterate over a list.

3. Change the following lines:

line 5 self.History[address] = set() #instead of [] (list)
line 7 self.History[address].add(media) #set uses add instead of
append

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why does list have no 'get' method?

[pruebauno]

On Feb 7, 12:15 pm, [EMAIL PROTECTED] wrote:
> On Feb 7, 11:01 am, "Denis Bilenko" <[EMAIL PROTECTED]> wrote:
>
>
>
> > Steve Holden wrote:
> > > These versions differ with respect to treatment of blank lines, which
> > > indicates  how easy it is to go astray in this kind of semantic
> > > optimization. Your example simply wouldn't work (though you could patch
> > > it up using "if line is None". (despite the use of short-circuiting
> > > predicates).
> > > > both examples show reduction by 3 lines.
>
> > > Perhaps so, but you have to realise that Python has never valued code
> > > compactness over clarity.
> > > I'm not sure that your democratic wish to ensure fairness to sequences
> > > will garner much support, interesting though it is in an academic sense.
>
> > Thank you for the patch. My incentives are not academic though.
>
> > I convinced that this
>
> > line = self._buffer.get(self._bufindex)
> > if line is None:
> > self._bufindex += 1
> > self._lineno += 1
> > self._filelineno += 1
> > return line
> > line = self.readline()
>
> > is more clear than
>
> > try:
> > line = self._buffer[self._bufindex]
> > except IndexError:
> > pass
> > else:
> > self._bufindex += 1
> > self._lineno += 1
> > self._filelineno += 1
> > return line
> > line = self.readline()
>
> > I mentioned 3 lines reduction just because it is
> > objective, while 'more clear' is subjective.
>
> > Code snippets are again not equivalent, e.g. in the
> > case when self._buffer[self._bufindex] exists and is
> > equal to None. Although the author could probably
> > make a guarantee that is never the case.
>
> This has been a pet peeve of mine too. So you are not completely
> crazy:
>
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/269484
>
> There is a discussion about precisely this by Alex Martelli in the
> printed version of the Python cookbook (2nd edition). I don't share
> the opinion but I am not going to fight the core Python developers
> over it. I don't think it is worth the effort. Python is so smooth
> already, there has to be some excuse to add some cute hack to your
> program once in a while :-).
>
> dict(enumerate(lst)).get(i,default) for example although not very
> efficient, looks cute. I think I like it better than my own solution.

Actually to clarify, I think this is more useful for tuples (or tuple
like structures) than lists per se. On homogeneous lists I usually
iterate. It is on heterogeneous that I want to extract specific fields
that might be optional.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why does list have no 'get' method?

[pruebauno]

On Feb 7, 11:01 am, "Denis Bilenko" <[EMAIL PROTECTED]> wrote:
> Steve Holden wrote:
> > These versions differ with respect to treatment of blank lines, which
> > indicates  how easy it is to go astray in this kind of semantic
> > optimization. Your example simply wouldn't work (though you could patch
> > it up using "if line is None". (despite the use of short-circuiting
> > predicates).
> > > both examples show reduction by 3 lines.
>
> > Perhaps so, but you have to realise that Python has never valued code
> > compactness over clarity.
> > I'm not sure that your democratic wish to ensure fairness to sequences
> > will garner much support, interesting though it is in an academic sense.
>
> Thank you for the patch. My incentives are not academic though.
>
> I convinced that this
>
> line = self._buffer.get(self._bufindex)
> if line is None:
> self._bufindex += 1
> self._lineno += 1
> self._filelineno += 1
> return line
> line = self.readline()
>
> is more clear than
>
> try:
> line = self._buffer[self._bufindex]
> except IndexError:
> pass
> else:
> self._bufindex += 1
> self._lineno += 1
> self._filelineno += 1
> return line
> line = self.readline()
>
> I mentioned 3 lines reduction just because it is
> objective, while 'more clear' is subjective.
>
> Code snippets are again not equivalent, e.g. in the
> case when self._buffer[self._bufindex] exists and is
> equal to None. Although the author could probably
> make a guarantee that is never the case.

This has been a pet peeve of mine too. So you are not completely
crazy:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/269484

There is a discussion about precisely this by Alex Martelli in the
printed version of the Python cookbook (2nd edition). I don't share
the opinion but I am not going to fight the core Python developers
over it. I don't think it is worth the effort. Python is so smooth
already, there has to be some excuse to add some cute hack to your
program once in a while :-).

dict(enumerate(lst)).get(i,default) for example although not very
efficient, looks cute. I think I like it better than my own solution.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why does list have no 'get' method?

[pruebauno]

On Feb 7, 4:36 pm, George Sakkis <[EMAIL PROTECTED]> wrote:
> On Feb 7, 4:25 pm, Steven D'Aprano <[EMAIL PROTECTED]
>
>
>
> cybersource.com.au> wrote:
> > On Thu, 07 Feb 2008 15:59:13 +0100, Wildemar Wildenburger wrote:
> > > Arnaud Delobelle wrote:
> > >> Personally, between
>
> > >> * foo if foo else bar
> > >> * foo or bar
>
> > >> I prefer the second. Maybe it could be spelt
>
> > >> * foo else bar ?
>
> > > How about
>
> > > val = foo rather than bar
>
> > > If that is not clear and obvios, I don't know what is.  ;)
>
> > In English, the expression "foo rather than bar" means "bar is never
> > chosen, foo is always chosen".
>
> Ok, the fix is easy:
>
> val = BETTER foo THAN bar
>
> ;-)
>
> Cobol-strikes-back-ly yours,
>
> George

I use a ETL language/tool that actually has a function for this kind
of thing:

NulltoValue(value,defaultValue)

it returns defaultValue if value is null otherwise value.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: optimizing large dictionaries

[pruebauno]

On Jan 15, 4:39 pm, Per Freem  wrote:
> hello
>
> i have an optimization questions about python. i am iterating through
> a file and counting the number of repeated elements. the file has on
> the order
> of tens of millions elements...
>
> i create a dictionary that maps elements of the file that i want to
> count
> to their number of occurs. so i iterate through the file and for each
> line
> extract the elements (simple text operation) and see if it has an
> entry in the dict:
>
> for line in file:
>   try:
>     elt = MyClass(line)# extract elt from line...
>     my_dict[elt] += 1
>   except KeyError:
>     my_dict[elt] = 1
>
> i am using try/except since it is supposedly faster (though i am not
> sure
> about this? is this really true in Python 2.5?).
>
> the only 'twist' is that my elt is an instance of a class (MyClass)
> with 3 fields, all numeric. the class is hashable, and so my_dict[elt]
> works well.
> the __repr__ and __hash__ methods of my class simply return str()
> representation
> of self, while __str__ just makes everything numeric field into a
> concatenated string:
>
> class MyClass
>
>   def __str__(self):
>     return "%s-%s-%s" %(self.field1, self.field2, self.field3)
>
>   def __repr__(self):
>     return str(self)
>
>   def __hash__(self):
>     return hash(str(self))
>
> is there anything that can be done to speed up this simply code? right
> now it is taking well over 15 minutes to process, on a 3 Ghz machine
> with lots of RAM (though this is all taking CPU power, not RAM at this
> point.)
>
> any general advice on how to optimize large dicts would be great too
>
> thanks for your help.

I am willing to bet a beer that the slow performance has nothing to do
with the dict but is either because of MyClass or the read from disk.
--
http://mail.python.org/mailman/listinfo/python-list

Re: is this pythonic?

[pruebauno]

On Jan 21, 12:34 pm, TP  wrote:
> alex23 wrote:
> > Try not to use 'dict' or the name of any of the other built-in types
>
> So my list is rather:
> l=[{"title":"to", "color":"blue", "value":2}
> {"title":"ti", "color":"red", "value":"coucou"}]
>
> So, I will rather use your solution:
>
> for index, record in enumerate(l):
>     if record['title'] == 'ti':
>         to_add_in_another_list = l.pop(index)
> another_list.append(to_add_in_another_list )
>
If you have duplicates this will not work. You will have to do
something like this instead:

>>> o=[]
>>> i=0
>>> ln=len(l)
>>> while i>> for index, record in enumerate(l):
if record['title'] == 'ti':
to_add_in_another_list = l.pop(index)
break

I don't know if these are more pythonic, they should be more efficient
for longer lists though.
--
http://mail.python.org/mailman/listinfo/python-list

Re: is this pythonic?

[pruebauno]

On Jan 21, 4:23 pm, Scott David Daniels  wrote:
> prueba...@latinmail.com wrote:
> > ... If you have duplicates this will not work. You will have to do
> > something like this instead:
>
>  o=[]
>  i=0
>  ln=len(l)
>  while i >    if l[i]['title']=='ti':
> >            o.append(l.pop(i))
> >            ln-=1
> >    else:
> >            i+=1
>
> Or the following:
>      indices = [i for i,d in enumerate(l) if d['title']=='ti']
>      for i in reversed(indices): # so del doesn't affect later positions
>          del l[i]
>
> --Scott David Daniels
> scott.dani...@acm.org

Cool. How come I didn't think of that! That means I can create an evil
one liner now >:-).

replacecount=len([o.append(l.pop(i)) for i in reversed(xrange(len(l)))
if l[i]['title']=='ti'])
--
http://mail.python.org/mailman/listinfo/python-list

Re: is this pythonic?

[pruebauno]

On Jan 21, 4:23 pm, Scott David Daniels  wrote:
> prueba...@latinmail.com wrote:
> > ... If you have duplicates this will not work. You will have to do
> > something like this instead:
>
>  o=[]
>  i=0
>  ln=len(l)
>  while i >    if l[i]['title']=='ti':
> >            o.append(l.pop(i))
> >            ln-=1
> >    else:
> >            i+=1
>
> Or the following:
>      indices = [i for i,d in enumerate(l) if d['title']=='ti']
>      for i in reversed(indices): # so del doesn't affect later positions
>          del l[i]
>
> --Scott David Daniels
> scott.dani...@acm.org

Cool. How come I didn't think of that!. Now I can write my evil one
liner >:-).

o=[l.pop(i) for i in reversed(xrange(len(l))) if l[i]['title']=='ti']
--
http://mail.python.org/mailman/listinfo/python-list

Re: funny generator behaviour

[pruebauno]

On Dec 4, 8:00 am, Edvin Fuglebakk <[EMAIL PROTECTED]> wrote:
> I have written a generator that puzzles me:
>
> The generator is supposed to create ordered selections of a set of
> objects. repetition of objects is allowed and the selections should be
> of a size determined by a pramter to the generator.
>
> Now, if I try to accummulate the generated selections into a list I get
> some peculiar behaviour that I hope maybe some of you can help me
> understand:
>
> Help much appreciated
> -Edvin
>
> #straightforward acumulation. Does not give the expected result
>  >>> d=[]
>  >>> for f in orderedCombinations([1,2],3):
> ... d.append(f)
> ...
>  >>> d
> [[1], [2], [1], [2], [1], [2], [1], [2]]
>
> #accumulating shallow copies of the genereated combinations works:
>  >>> d=[]
>  >>> for f in orderedCombinations([1,2],3):
> ... d.append(f[:])
> ...
>  >>> d
> [[1, 1, 1], [1, 1, 2], [1, 2, 1], [1, 2, 2], [2, 1, 1], [2, 1, 2], [2,
> 2, 1], [2, 2, 2]]
>
> #The generator:
> def orderedCombinations(pool, k):
> """
> Generator yielding ordered selections of size k with repetition from
> pool.
> """
>
> if k == 1:
> for m in pool:
> yield [m]
>
> if k > 1:
>
> for m in pool:
> for combo in orderedCombinations(pool, k-1):
>
> #insert and pop to avoid copying entire list
> combo.insert(0,m)
> yield combo
> combo.pop(0)

Functional style (like the recursion you are using) and mutable
datastructures are hard to reason about. This version works for me:

def orderedCombinations(pool, k):
"""
Generator yielding ordered selections of size k with repetition
from
pool.
"""

if k == 1:
for m in pool:
yield [m]

if k > 1:

for m in pool:
for combo in orderedCombinations(pool, k-1):
yield [m]+combo
--
http://mail.python.org/mailman/listinfo/python-list

Re: funny generator behaviour

[pruebauno]

On Dec 4, 8:00 am, Edvin Fuglebakk <[EMAIL PROTECTED]> wrote:
> I have written a generator that puzzles me:
>
> The generator is supposed to create ordered selections of a set of
> objects. repetition of objects is allowed and the selections should be
> of a size determined by a pramter to the generator.
>
> Now, if I try to accummulate the generated selections into a list I get
> some peculiar behaviour that I hope maybe some of you can help me
> understand:
>
> Help much appreciated
> -Edvin
>
> #straightforward acumulation. Does not give the expected result
>  >>> d=[]
>  >>> for f in orderedCombinations([1,2],3):
> ... d.append(f)
> ...
>  >>> d
> [[1], [2], [1], [2], [1], [2], [1], [2]]
>
> #accumulating shallow copies of the genereated combinations works:
>  >>> d=[]
>  >>> for f in orderedCombinations([1,2],3):
> ... d.append(f[:])
> ...
>  >>> d
> [[1, 1, 1], [1, 1, 2], [1, 2, 1], [1, 2, 2], [2, 1, 1], [2, 1, 2], [2,
> 2, 1], [2, 2, 2]]
>
> #The generator:
> def orderedCombinations(pool, k):
> """
> Generator yielding ordered selections of size k with repetition from
> pool.
> """
>
> if k == 1:
> for m in pool:
> yield [m]
>
> if k > 1:
>
> for m in pool:
> for combo in orderedCombinations(pool, k-1):
>
> #insert and pop to avoid copying entire list
> combo.insert(0,m)
> yield combo
> combo.pop(0)

BTW if you search in the fine manual version 2.6 you will find that a
slick implementation is already part of the standard library (hint:
itertools):

def orderedCombinations(pool, k):
result = [[]]
for poolel in [pool] * k:
result = [x+[y] for x in result for y in poolel]
return result
--
http://mail.python.org/mailman/listinfo/python-list

Re: funny generator behaviour

[pruebauno]

On Dec 4, 8:00 am, Edvin Fuglebakk <[EMAIL PROTECTED]> wrote:
> I have written a generator that puzzles me:
>
> The generator is supposed to create ordered selections of a set of
> objects. repetition of objects is allowed and the selections should be
> of a size determined by a pramter to the generator.
>
> Now, if I try to accummulate the generated selections into a list I get
> some peculiar behaviour that I hope maybe some of you can help me
> understand:
>
> Help much appreciated
> -Edvin
>
> #straightforward acumulation. Does not give the expected result
>  >>> d=[]
>  >>> for f in orderedCombinations([1,2],3):
> ... d.append(f)
> ...
>  >>> d
> [[1], [2], [1], [2], [1], [2], [1], [2]]
>
> #accumulating shallow copies of the genereated combinations works:
>  >>> d=[]
>  >>> for f in orderedCombinations([1,2],3):
> ... d.append(f[:])
> ...
>  >>> d
> [[1, 1, 1], [1, 1, 2], [1, 2, 1], [1, 2, 2], [2, 1, 1], [2, 1, 2], [2,
> 2, 1], [2, 2, 2]]
>
> #The generator:
> def orderedCombinations(pool, k):
> """
> Generator yielding ordered selections of size k with repetition from
> pool.
> """
>
> if k == 1:
> for m in pool:
> yield [m]
>
> if k > 1:
>
> for m in pool:
> for combo in orderedCombinations(pool, k-1):
>
> #insert and pop to avoid copying entire list
> combo.insert(0,m)
> yield combo
> combo.pop(0)

def orderedCombinations(pool, k):
res=[[]]
for i in xrange(k):
res=[n+[m] for n in res for m in pool]
return res
--
http://mail.python.org/mailman/listinfo/python-list

Re: Good introductory book?

[pruebauno]

On Dec 3, 9:44 am, "Ken D'Ambrosio" <[EMAIL PROTECTED]> wrote:
> Hi, all.  I'm getting ready to do some projects in Python, and I've cut my
> teeth a little bit, but I've found the "Learning|Programming Python" books
> from O'Reilly to be more-or-less useless (to my surprise -- I'm usually an
> O'Reilly fan).  I really, really like "Python Essential Reference", but
> it's -- well, more of a reference than an intro.  So, an introductory text
> that actually assumes some previous programming experience (as opposed to
> "Learning Python" which must be the most slowly-paced programming book
> ever) would be terrific.
>
> Thanks for your suggestions!
>
> -Ken

I had the same experience as you had and almost gave up on Python. I
had programmed in other languages before and found the Programming
Python book very tedious to read. I just wanted to learn the syntax
and library to start writing my own stuff. Luckily, I found Guido's 12
page tutorial (http://www.python.org/doc/2.5.2/tut/tut.html) and that
plus the online library reference and Google was everything I ever
needed since then. That said I looked at O'Reilly's "Python in a
Nutshell" and I thought it was really good. It is a mix between
introduction, language and library reference. I should have gotten
that instead of Learning/Programming when I started and I probably
would have been much happier.
--
http://mail.python.org/mailman/listinfo/python-list

Re: StringIO in 2.6 and beyond

[pruebauno]

On Dec 9, 11:28 am, Bill McClain
<[EMAIL PROTECTED]> wrote:
> On 2008-12-08, Bill McClain <[EMAIL PROTECTED]> wrote:
>
> > On 2008-12-08, Christian Heimes <[EMAIL PROTECTED]> wrote:
> > > In this context 'str' means Python 3.0's str type, which is unicode in
> > > 2.x. Please report the misleading error message.
> > So this is an encoding problem? Can you give me a hint on how to correct in 
> > my
> > example? I see that io.StringIO() has an encoding parameter, but I'm unclear
> > what to specify.
>
> I still don't have this working. I've been specifying encodings without
> success.
>
> The StringIO example usage in the Python 3.0 documentation here:
>
>      http://docs.python.org/3.0/library/io.html#io.StringIO
>
> gives me the same error on 2.6:
>
>     #! /usr/bin/env python
>
>     from __future__ import print_function
>     import io
>
>     output = io.StringIO()
>     output.write('First line.\n')
>     print('Second line.', file=output)
>
>     # Retrieve file contents -- this will be
>     # 'First line.\nSecond line.\n'
>     contents = output.getvalue()
>
>     # Close object and discard memory buffer --
>     # .getvalue() will now raise an exception.
>     output.close()
>
> ./stringio30.py
> Traceback (most recent call last):
>   File "./stringio30.py", line 7, in 
>     output.write('First line.\n')
>   File "/usr/local/lib/python2.6/io.py", line 1487, in write
>     s.__class__.__name__)
> TypeError: can't write str to text stream
>
> -Bill
> --
> Sattre Press                              History of 
> Astronomyhttp://sattre-press.com/              During the 19th Century
> [EMAIL PROTECTED]                       by Agnes M. Clerke
>                              http://sattre-press.com/han.html

This puzzles me too. According to the documentation StringIO accepts
both byte strings and unicode strings. Try to replace
   output.write('First line.\n')
with
   output.write(unicode('First line.\n'))
or
   output.write(str('First line.\n'))
and see if one of those works.
--
http://mail.python.org/mailman/listinfo/python-list

Re: When (and why) to use del?

[pruebauno]

On Dec 9, 11:35 am, Albert Hopkins <[EMAIL PROTECTED]> wrote:
> I'm looking at a person's code and I see a lot of stuff like this:
>
>         def myfunction():
>             # do some stuff stuff
>             my_string = function_that_returns_string()
>             # do some stuff with my_string
>             del my_string
>             # do some other stuff
>             return
>
> and also
>
>         def otherfunction():
>             try:
>                 # some stuff
>             except SomeException, e:
>                 # more stuff
>                 del e
>             return
>
> I think this looks ugly, but also does it not hurt performance by
> preempting the gc?  My feeling is that this is a misuse of 'del'. Am I
> wrong?  Is there any advantage of doing the above?

If this is CPython (the standard Python) the extra del(s) in the
examples you gave are just garbage and good for nothing. The reason
being that they are inside a function; as soon as the program exits
the function the variable gets deleted automatically. The only reason
to use del would be a global variable in long running program that
uses a lot of space, but global variables should be avoided for other
reasons anyway. I never ever used del in my programs only on the
interactive interpreter prompt after doing foolish things like x=open
('1GBfile').read()
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3 For Python 2 Users

[pruebauno]

On Dec 9, 11:58 am, Tim Daneliuk <[EMAIL PROTECTED]> wrote:
> I code in Python 2.x intermittently and have only casually watched the
> 3.x development discussions.  Now it's time to get up to speed.
> Has someone written a tutorial for people in my situation.  Yes, I've
> looked at the release notes, but I'm looking for something that
> motivates all the major changes to give me a better sense of the
> Gestalt of the new language.
>
> TIA,
> --
> 
> Tim Daneliuk     [EMAIL PROTECTED]
> PGP Key:        http://www.tundraware.com/PGP/

For starters this should work:

http://docs.python.org/dev/3.0/whatsnew/3.0.html
--
http://mail.python.org/mailman/listinfo/python-list

Re: StringIO in 2.6 and beyond

[pruebauno]

On Dec 10, 6:58 am, Bill McClain
<[EMAIL PROTECTED]> wrote:
> On 2008-12-10, ajaksu <[EMAIL PROTECTED]> wrote:
>
> > On Dec 9, 5:24 pm, Bill McClain <[EMAIL PROTECTED]>
> > wrote:
> > > On 2008-12-09, MRAB <[EMAIL PROTECTED]> wrote:
>
> > > > In Python 2.x unmarked string literals are bytestrings. In Python 3.x
> > > > they're Unicode. The intention is to make the transition from 2.x to 3.x
> > > > easier by adding some features of 3.x to 2.x, but without breaking
> > > > backwards compatibility (not entirely successfully!).
>
> > > It is a bit ugly. In 2.6 StringIO won't take bytestrings, so I apply 
> > > u'x'. But
> > > in 3.0 u'x' will be gone and I'll have to change the code again.
> > Try:
> > from __future__ import unicode_literals
>
> That works for:
>
>     output.write('First line.\n')
>
> ...but not for:
>
>    print('Second line.', file=output)
>
> Maybe a combination of this and functools.partial as was suggested before. At
> least the necessary edits would be at the top of the program.
>
> -Bill
> --
> Sattre Press                                      Tales of 
> Warhttp://sattre-press.com/                      by Lord Dunsany
> [EMAIL PROTECTED]        http://sattre-press.com/tow.html

I think this combination might do the trick (I don't have 2.6 to test
it right now):

from __future__ import print_function
from __future__ import unicode_literals
from functools import partial
import io
print = partial(print, sep=" ", end="\n")
out = io.StringIO()
print("hello", file=out)

What puzzles me is the documentation in 2.6 and 3.0:
In 2.6 it says: "The StringIO object can accept either Unicode or 8-
bit strings". Why does it fail with old str objects then?
Why is there no documentation for StringIO in 3.0?
--
http://mail.python.org/mailman/listinfo/python-list

Re: StringIO in 2.6 and beyond

[pruebauno]

On Dec 10, 10:06 am, [EMAIL PROTECTED] wrote:
> On Dec 10, 6:58 am, Bill McClain
>
>
>
> <[EMAIL PROTECTED]> wrote:
> > On 2008-12-10, ajaksu <[EMAIL PROTECTED]> wrote:
>
> > > On Dec 9, 5:24 pm, Bill McClain <[EMAIL PROTECTED]>
> > > wrote:
> > > > On 2008-12-09, MRAB <[EMAIL PROTECTED]> wrote:
>
> > > > > In Python 2.x unmarked string literals are bytestrings. In Python 3.x
> > > > > they're Unicode. The intention is to make the transition from 2.x to 
> > > > > 3.x
> > > > > easier by adding some features of 3.x to 2.x, but without breaking
> > > > > backwards compatibility (not entirely successfully!).
>
> > > > It is a bit ugly. In 2.6 StringIO won't take bytestrings, so I apply 
> > > > u'x'. But
> > > > in 3.0 u'x' will be gone and I'll have to change the code again.
> > > Try:
> > > from __future__ import unicode_literals
>
> > That works for:
>
> >     output.write('First line.\n')
>
> > ...but not for:
>
> >    print('Second line.', file=output)
>
> > Maybe a combination of this and functools.partial as was suggested before. 
> > At
> > least the necessary edits would be at the top of the program.
>
> > -Bill
> > --
> > Sattre Press                                      Tales of 
> > Warhttp://sattre-press.com/                     by Lord Dunsany
> > [EMAIL PROTECTED]        http://sattre-press.com/tow.html
>
> I think this combination might do the trick (I don't have 2.6 to test
> it right now):
>
> from __future__ import print_function
> from __future__ import unicode_literals
> from functools import partial
> import io
> print = partial(print, sep=" ", end="\n")
> out = io.StringIO()
> print("hello", file=out)
>
> What puzzles me is the documentation in 2.6 and 3.0:
> In 2.6 it says: "The StringIO object can accept either Unicode or 8-
> bit strings". Why does it fail with old str objects then?
> Why is there no documentation for StringIO in 3.0?

OK I found StringIO it is called io.StringIO now.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Removing None objects from a sequence

[pruebauno]

On Dec 12, 8:08 am, "Filip Gruszczyński"  wrote:
> I am not doing it, because I need it. I can as well use "if not elem
> is None", but I just wanted to know, if there is some better way of
> doing this. I like to know :-)
>
> And I can't understand why you are becoming so aggressive because of
> this. Just because I asked for that, doesn't mean, that I will put
> some obfuscated code into my project. I just wanted to learn something
> new - I checked itertools, I googled a bit, now I wanted to ask here
> if someone knew some really cool way of this.  All the other
> assumptions weren't really necessary.
>
> Thanks for those ideas, however. I like the last one a lot :)
>
> --
> Filip Gruszczyński

In this case the "cool" way is the straightforward way:
either the list versions:

[x fox x in seq if x is not None]
filter(lambda x: x is not None, seq)

or the generator versions:

(x for x in seq if x is not None)
itertools.ifilter(lambda x: x is not None, seq)

--
http://mail.python.org/mailman/listinfo/python-list

Re: stable algorithm with complexity O(n)

[pruebauno]

> Non-comparison sorts are a useful technique, but it's changing the
> problem, and they are only useful in very limited circumstances. There's
> a good reason that most sort routines are based on O(n*log n) comparison
> sorts instead of O(n) bucket sorts or radix sorts.
>
This is an assumption that I never quite understood. What most people
want is to have sorted data, they don't care if I used a sorting or
non-sorting comparison to do it. I think it is just that in most cases
n is not very big anyway and comparison sorts make it easier on the
programmer to create arbitrary types that are sortable.
--
http://mail.python.org/mailman/listinfo/python-list

Re: stable algorithm with complexity O(n)

[pruebauno]

On Dec 15, 11:05 am, prueba...@latinmail.com wrote:
> > Non-comparison sorts are a useful technique, but it's changing the
> > problem, and they are only useful in very limited circumstances. There's
> > a good reason that most sort routines are based on O(n*log n) comparison
> > sorts instead of O(n) bucket sorts or radix sorts.
>
> This is an assumption that I never quite understood. What most people
> want is to have sorted data, they don't care if I used a sorting or
> non-sorting comparison to do it. I think it is just that in most cases
> n is not very big anyway and comparison sorts make it easier on the
> programmer to create arbitrary types that are sortable.

I meant they don't care if I use a comparison or non-comparison sort
of course.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Why no lexical scoping for a method within a class?

[pruebauno]

On Dec 17, 10:19 am, walterbyrd  wrote:
> For a language as well structured as Python, this seems somewhat
> sloppy, and inconsistant.  Or is there some good reason for this?
>
> Here is what I mean:
>
> def a():
>     x = 99
>     print x
>
> def b():
>     print x
>
> a()
> b() # raises an exception because x is not defined.
>
> However in the methods are within a class, the scoping seems to work
> differently.
>
> class ab():
>     def a(self):
>         self.x = 99
>         print self.x
>     def b(self):
>         print self.x
>
> i = ab()
> i.a()
> i.b() # this works, why no lexical scoping?

If scoping worked as you want, how, pray tell, would you define object
attributes?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is this pythonic?

[pruebauno]

On Dec 18, 11:08 am, ipyt...@gmail.com wrote:
> x.validate_output(x.find_text(x.match_filename
> (x.determine_filename_pattern(datetime.datetime.now()
>
> Is it even good programming form?

Lisp and Scheme programmers love that style. You can tell by the
number of parentheses :-). In Python people usually use an
intermediate variable to break things up a bit but the amount of
acceptable nesting is a matter of personal style.
--
http://mail.python.org/mailman/listinfo/python-list

Re: 2to3 used in the Shootout

[pruebauno]

On Dec 23, 5:21 pm, Isaac Gouy  wrote:
> On Dec 23, 11:51 am, bearophileh...@lycos.com wrote:
>
> > They have translated the Python benchmarks of the Shootout site from
> > Py2 to Py3 using 2to3:
>
> >http://shootout.alioth.debian.org/u32/benchmark.php?test=all〈=pyt...
>
> So please re-write those programs to remove problems created by
> automatic translation and better take advantage of Python 3
> functionality...
>
> http://shootout.alioth.debian.org/u32/faq.php#play
>
> > It shows some "performance bugs" of Python3 itself (especially
> > regarding the binary-trees benchmark, that was unexpected by me), and
> > two points where 2to3 may be improved, for example after the
> > translation this gives error:
> >          table=string.maketrans('ACBDGHK\nMNSRUTWVYacbdghkmnsrutwvy',
> >                                 'TGVHCDM
> > \nKNSYAAWBRTGVHCDMKNSYAAWBR')):
>
> > Gives:
> > TypeError: maketrans arguments must be bytes objects
>
> > Bye,
> > bearophile
>
>
BTW I am not sure how to submit this or if this is actually valid to
do, but I have a faster version for the pidigits program that uses
basically the same algorithm but removes function calls and unused
terms of the formula.


import time

def pi_digits(n, width):
out = []
wrt = out.append
aq = 1
ar = 0
at = 1
k = 0
f = 1
g = 2
i = 0
while i < n:
y = (aq*3+ar)//at
while y != ((aq*4+ar)//at):
k += 1
f += 2
g += 4
ar = aq*g+ar*f
aq = aq*k
at = at*f
y = (aq*3+ar)//at
aq = 10*aq
ar = 10*ar-10*y*at
i += 1
wrt(str(y))
if not i%width:
wrt('\t:%d\n'%i)
wrt(' '*(width-i%width))
wrt('\t:%d\n'%i)
return ''.join(out)


def main():
begin = time.time()
n = 1000
width = 70
print pi_digits(n,width)
print 'Total Time:', time.time()-begin

main()


--
http://mail.python.org/mailman/listinfo/python-list

Re: SQL, lite lite lite

[pruebauno]

On Dec 29, 1:06 pm, Aaron Brady  wrote:
> Hi all,
>
> About a year ago, I posted an idea I was having about thread
> synchronization to the newsgroup.  However, I did not explain it well,
> and I really erred on the side of brevity.  (After some finagling, Mr.
> Bieber and I decided it wasn't exactly anything groundbreaking.)  But
> I think the brevity cost me some readers, who might have had more
> interest.  The affair was on the whole discouraging.  So, I'm going to
> try another idea, and assume that readers have some time, and will
> spend it on it.
>
> I don't think relational data can be read and written very easily in
> Python.  There are some options, such as 'sqllite3', but they are not
> easy.  'sqllite3' statements are valid SQL expressions, which afford
> the entire power of SQL, but contrary to its name, it is not that
> 'lite'.  To me, 'lite' is something you could learn (even make!) in an
> afternoon, not a semester; something the size of an ActiveState
> recipe, or a little bigger, maybe a file or two.  If you think SQL is
> a breeze, you probably won't find my idea exciting.  I assume that the
> basics of SQL are creating tables, selecting records, and updating
> records.
>
> My idea is to create a 'Relation' class.  The details are basically
> open, such as whether to back it with 'sqllite3', 'shelve', 'mmap', or
> just mapping and sequence objects; what the simplest syntax is that
> can capture and permit all the basics, and how much and what else can
> fit in at that level; how and whether it can include arbitrary Python
> objects, and what constraints there are on them if not; how and
> whether to permit transactions; and what the simplest and coolest
> thing you can do with a little Python syntax is.
>
> This is basically an invitation for everyone to brainstorm.  (No
> hijackings, good humor & digression ok.)  Lastly, ...
>
> **warning, spoiler!  here's what I thought of already.**
>
> **repeat!  spoiler!  here's what I thought of already.**
>
> #Just the select and update syntax:
>
> >>> a= people._select( "firstname== 'Joe'" )
>
> #select 'key' from 'people' where 'firstname'== 'joe'>>> a
>
> [Entry2864, Entry3076, Entry3172]>>> entry1= a[ 0 ]
> >>> entry1.phone
>
> #select 'phone' from 'people' where 'key'==self.key
> "555-2413">>> entry1.phone= "555-1234"
>
> #update 'people' set 'phone'= '555-1234' where 'key'==self.key>>> entry1.phone
>
> "555-1234"
>
> #Create table syntax (a-whole-nother beast in itself):
>
> >>> classes= db.Relation( 'class_', 'person', Unique( 'class_', 'person' ) )
>
> #create table 'classes' ( 'key', 'class_', 'person' ) unique
> ( 'class_', 'person' )
>
> >>> classes._unique( 'class_', 'person' )
> >>> classes.class_.noneok= False #'class_' cannot be null
> >>> classes.person.noneok= False
> >>> classes._insert( 'Physics', 'Dan' )
> >>> classes._insert( 'Chem', 'Tim' )
>
> Hoping-"good critic"-is-self-consistent-ly, hoping-to-hear-it's-too-
> complicated-already-ly,
> A. Brady

You really do like to reinvent the wheels do you? :-) Nothing wrong
with that. Just be aware that most people that really need what you
are proposing are probably already using mature feature rich libraries
for that.

http://wiki.python.org/moin/HigherLevelDatabaseProgramming
--
http://mail.python.org/mailman/listinfo/python-list

Re: f.seek() unwanted output

[pruebauno]

On Jan 5, 8:52 am, "thomasvang...@gmail.com" 
wrote:
> I'm having trouble with a script that is printing the output of f.seek
> () whereas in the documentation it is quoted not to have any output:
>
> 
> file.seek(offset[, whence])¶
>
>     Set the file’s current position, like stdio‘s fseek. The whence
> argument is optional and defaults to os.SEEK_SET or 0 (absolute file
> positioning); other values are os.SEEK_CUR or 1 (seek relative to the
> current position) and os.SEEK_END or 2 (seek relative to the file’s
> end). There is no return value.
> --
>
> I have a file in memory.
> when i try f.seek(0) #or any other value in f.tell()
> it gives me 0 as output:
>
> the following script illustrates my 'problem'>>> for a in range(10):
>
>         f.seek(a)
>
> 0
> 1
> 2
> 3
> 4
> 5
> 6
> 7
> 8
> 9
>
>
>
> I don't want python to produce output when setting the file pointer.
> Any help woul be appreciated.
> Kind regards,
> Thomas

You can also avoid the output by assigning the output to something:

>>> for a in range(10):
 dummy=f.seek(a)

--
http://mail.python.org/mailman/listinfo/python-list

Re: Any news on when some libraries will be ported to Python 3.0?

[pruebauno]

On Jan 7, 3:48 am, "Just Another Victim of the Ambient Morality"
 wrote:
>     Anyway, I'd love to hear some news about any of these things in
> particular or even anything in general.  Am I the only one who's psyched for
> this version of Python?
>     Thank you...

There are many people psyched about 3.0, but also many older farts
that have been using Python since 1.0 and have tons of code that
already works fine and feel a little bit like the mechanic that has to
switch his tools from imperial to metric; it is overall probably for
the better but still a pain to adapt everything to the new system.
This is open source so you either have to find a way to motivate the
authors of the libraries without pissing them off, or you will have to
be patient.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Encrypted Logging in python

[pruebauno]

On Jan 9, 8:02 am, koranth...@gmail.com wrote:
> Also, what I am asking is a generic option in logging - which can help
> the adoption of the logging framework in even closed source systems.
> It is not just about security - just that a closed source company
> might be much more comfortable in using the system if crypt is there.

Python is an open source project. Many people that read this list
don't like closed source code too much and are not willing to invest
time to work in features like this. You might get lucky and somebody
that is interested in the topic might give you some tips.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Implementing file reading in C/Python

[pruebauno]

On Jan 9, 8:48 am, Johannes Bauer  wrote:
> No - and I've not known there was a profiler yet have found anything
> meaningful (there seems to be an profiling C interface, but that won't
> get me anywhere). Is that a seperate tool or something? Could you
> provide a link?
> Thanks,
> Kind regards,
> Johannes

It is part of the python standard library:

http://docs.python.org/library/profile.html
--
http://mail.python.org/mailman/listinfo/python-list

Re: Dict Comprehension ?

[pruebauno]

On Oct 6, 8:59 am, "Ernst-Ludwig Brust" <[EMAIL PROTECTED]> wrote:
> Given 2 Number-Lists say l0 and l1,
> count the various positiv differences between the 2 lists
>
> the following part works:
>
> dif=[abs(x-y) for x in l0 for y in l1]
> da={}
> for d in dif: da[d]=da.get(d,0)+1
>
> i wonder, if there is a way, to avoid the list dif
>
> Ernst-Ludwig Brust

from collections import defaultdict
da=defaultdict(int)
for x in [10]:
for y in [11]:
da[abs(x-y)]+=1
--
http://mail.python.org/mailman/listinfo/python-list

Re: Builing Python 2.6 on AIX 5.2

[pruebauno]

On Oct 6, 4:16 am, brasse <[EMAIL PROTECTED]> wrote:
> Hello!
>
> I am having some trouble building Python 2.6 on AIX. The steps I have
> taken are:
>
> export PATH=/usr/bin/:/usr/vacpp/bin/
> ./configure --with-gcc=xlc_r --with-cxx=xlC_r --disable-ipv6
> make
>
> This is the error message I'm seeing:
> ./Modules/ld_so_aix xlc_r -bI:Modules/python.exp build/
> temp.aix-5.2-2.6/home/mabr/Python-2.6/Modules/_multiprocessing/
> multiprocessing.o build/temp.aix-5.2-2.6/home/mabr/Python-2.6/Modules/
> _multiprocessing/socket_connection.o build/temp.aix-5.2-2.6/home/mabr/
> Python-2.6/Modules/_multiprocessing/semaphore.o -L/usr/local/lib -o
> build/lib.aix-5.2-2.6/_multiprocessing.so
> ld: 0711-317 ERROR: Undefined symbol: .sem_timedwait
> ld: 0711-317 ERROR: Undefined symbol: .CMSG_SPACE
> ld: 0711-317 ERROR: Undefined symbol: .CMSG_LEN
> ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
> information.
> *** WARNING: renaming "_multiprocessing" since importing it failed: No
> such file or directory
> error: No such file or directory
> make: The error code from the last command is 1.
>
> Have someone on this list had similar problems? Am I missing some
> libraries? The configure script runs without errors, I would have
> expected some kind of error there if I was missing something.
>
> Regards,
> Mattias

I don't know if it has anything to do with it. But I got that error
one time on AIX and it was a library compiled to 32 bit that could not
be linked to 64 bit python. Maybe in your case it is the other way
around.

I haven't tried compiling Py2.6 yet. I compile 64 bit with:

OPT=-O2 ./configure --with-gcc="xlc_r -q64" --with-cxx="xlC_r -q64" --
disable-ipv6 AR="ar -X64"
--
http://mail.python.org/mailman/listinfo/python-list

Re: Builing Python 2.6 on AIX 5.2

[pruebauno]

On Oct 6, 11:03 am, "Jesse Noller" <[EMAIL PROTECTED]> wrote:
> Looks like AIX is missing sem_timedwait - see:http://bugs.python.org/issue3876
>
> Please add your error to the bug report just so I can track it.
>
> -jesse
>
> On Mon, Oct 6, 2008 at 4:16 AM, brasse <[EMAIL PROTECTED]> wrote:
> > Hello!
>
> > I am having some trouble building Python 2.6 on AIX. The steps I have
> > taken are:
>
> > export PATH=/usr/bin/:/usr/vacpp/bin/
> > ./configure --with-gcc=xlc_r --with-cxx=xlC_r --disable-ipv6
> > make
>
> > This is the error message I'm seeing:
> > ./Modules/ld_so_aix xlc_r -bI:Modules/python.exp build/
> > temp.aix-5.2-2.6/home/mabr/Python-2.6/Modules/_multiprocessing/
> > multiprocessing.o build/temp.aix-5.2-2.6/home/mabr/Python-2.6/Modules/
> > _multiprocessing/socket_connection.o build/temp.aix-5.2-2.6/home/mabr/
> > Python-2.6/Modules/_multiprocessing/semaphore.o -L/usr/local/lib -o
> > build/lib.aix-5.2-2.6/_multiprocessing.so
> > ld: 0711-317 ERROR: Undefined symbol: .sem_timedwait
> > ld: 0711-317 ERROR: Undefined symbol: .CMSG_SPACE
> > ld: 0711-317 ERROR: Undefined symbol: .CMSG_LEN
> > ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more
> > information.
> > *** WARNING: renaming "_multiprocessing" since importing it failed: No
> > such file or directory
> > error: No such file or directory
> > make: The error code from the last command is 1.
>
> > Have someone on this list had similar problems? Am I missing some
> > libraries? The configure script runs without errors, I would have
> > expected some kind of error there if I was missing something.
>
> > Regards,
> > Mattias
> > --
> >http://mail.python.org/mailman/listinfo/python-list

FYI
I read that some people use:
HAVE_BROKEN_POSIX_SEMAPHORES in pyconfig.h.

for AIX, although I never do. Maybe I am not doing anything advanced
enough for problems related to semaphores to show up.
--
http://mail.python.org/mailman/listinfo/python-list

Re: python 3: sorting with a comparison function

[pruebauno]

On Oct 10, 8:35 am, Kay Schluehr <[EMAIL PROTECTED]> wrote:
> On 9 Okt., 22:36, [EMAIL PROTECTED] wrote:
>
> > Yes, that's a wonderful thing, because from the code I see around
> > 99.9% of people see the cmp and just use it, totally ignoring the
> > presence of the 'key' argument, that allows better and shorter
> > solutions of the sorting problem.
>
> Me too because I don't get this:
>
> "key specifies a function of one argument that is used to extract a
> comparison key from each list element: key=str.lower. The default
> value is None."
>
> Kay

Don't know if further explanation is needed, but here is the deal:

cmp is a function that receives two values and you return -1, 0 or 1
depending if the first is smaller, equal or bigger. 99% of the time
you will do some operation on the values that come in and then do a if
statement with ">" or "<" and return -1,0,1.

key is a function that receives one value and you return the value
that you would normally compare against.

Let me show an example:

>>> data=[(4,'v'),(2,'x'),(1,'a')]
>>> sorted(data)
[(1, 'a'), (2, 'x'), (4, 'v')]

OK, we sorted the data, but What if we want to sort by the letter
instead of the number? Let's use cmp:

>>> def comp(x, y):
  key_of_x=x[1]
  key_of_y=y[1]
  if key_of_x < key_of_y:
return -1
  elif key_of_x > key_of_y:
return 1
  else:
return 0 #key_of_x == key_of_y

>>> sorted(data,cmp=comp)
[(1, 'a'), (4, 'v'), (2, 'x')]

Very well, so how do we do this using key?

>>> def keyfunc(x):
  key_of_x=x[1]
  return key_of_x

>>> sorted(data,key=keyfunc)
[(1, 'a'), (4, 'v'), (2, 'x')]


Same output. Very good.

(Of course a smart python developer would use the operator module so
he doesn't even have to write keyfunc but this was just an example)

In summary to transform most cmp functions to a key function you just
take the code that calculates the first value to be compared and leave
out the rest of the logic.

Hope that was helpful.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Where/how to propose an addition to a standard module?

[pruebauno]

On Oct 10, 3:10 pm, Joe Strout <[EMAIL PROTECTED]> wrote:
> I would like to propose a new method for the string.Template class.
> What's the proper procedure for doing this?  I've joined the python-
> ideas list, but that seems to be only for proposed language changes,
> and my idea doesn't require any change to the language at all.
>
>  From , it sounds like the
> PEP process is appropriate here, though other PEPs (like 
>   > make it sound as though these are meant for proposals for "Python
> 3000", which is not necessarily my intent.
>
> Here's a brief sketch of my proposal, in case it helps:
>
> Add a "match" function to string.Template, which takes a text string
> as a parameter.  If this text string can be matched to the template,
> by substituting some portion of the given string for each field of the
> template, then .match returns a dictionary, where each key is a field
> name and the value is the corresponding text from the input.  If the
> text string cannot be matched to the template, then .match returns None.
>
> I understand that if I'm to write a PEP, I'll need to flesh this out
> considerably as per PEP 0001.  But that document also suggests first
> discussing it here.  I'm still a newbie (or actually, oldbie-turned-
> nonbie-turned-newbie-again), so I could use some advice.  What's the
> next step in advocating for this idea?
>
> Thanks,
> - Joe

Whenever I needed such functionality I used the re module. The benefit
is that it uses unix style regular expression syntax and an egrep/awk/
perl/ruby user can understand it. You should show a few examples where
your proposal looks better than just using RE.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Append a new value to dict

[pruebauno]

On Oct 13, 9:41 am, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote:
> On Mon, 13 Oct 2008 14:10:43 +0200, Mathias Frey wrote:
> > However incrementing a non-existing key throws an exception. So you
> > either have to use a workaround:
>
> >  >>> try:
> > ...   counter['B'] += 1
> > ... except KeyError:
> > ...   counter['B'] = 1
>
> > Since this looks ugly somebody invented the setdefault method:
>
> >  >>> counter['B'] = counter.setdefault('B',0) + 1
>
> Nope, for this use case there is the `dict.get()` method:
>
> counter['B'] = counter.get('B', 0) + 1
>
> This assigns only *once* to ``counter['B']`` in every case.
>
> `dict.setdefault()` is for situations where you really want to actually
> put the initial value into the dictionary, like with the list example by
> the OP.
>
> Ciao,
> Marc 'BlackJack' Rintsch

...and if you are using Python 2.5 or later you can use the
collections module with collections.defaultdict(list) or
collections.defaultdict(int) to do the same thing. I personally find
it easier to read.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Possible read()/readline() bug?

[pruebauno]

On Oct 22, 2:54 pm, Mike Kent <[EMAIL PROTECTED]> wrote:
> Before I file a bug report against Python 2.5.2, I want to run this by
> the newsgroup to make sure I'm not being stupid.
>
> I have a text file of fixed-length records I want to read in random
> order.  That file is being changed in real-time by another process,
> and my process want to see the changes to the file.  What I'm seeing
> is that, once I've opened the file and read a record, all subsequent
> seeks to and reads of that same record will return the same data as
> the first read of the record, so long as I don't close and reopen the
> file.  This indicates some sort of buffering and caching is going on.
>
> Consider the following:
>
> $ echo "hi" >foo.txt  # Create my test file
> $ python2.5  # Run Python
> Python 2.5.2 (r252:60911, Sep 22 2008, 16:13:07)
> [GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>
> >>> f = open('foo.txt')  # Open my test file
> >>> f.seek(0)# Seek to the beginning of the file
> >>> f.readline() # Read the line, I get the data I expected
> 'hi\n'
> >>> # At this point, in another shell I execute 'echo "bye" >foo.txt'.  
> >>> 'foo.txt' now has been changed
> >>> # on the disk, and now contains 'bye\n'.
> >>> f.seek(0)# Seek to the beginning of the still-open file
> >>> f.readline() # Read the line, I don't get 'bye\n', I get the 
> >>> original data, which is no longer there.
> 'hi\n'
> >>> f.close() # Now I close the file...
> >>> f = open('foo.txt') # ... and reopen it
> >>> f.seek(0)   # Seek to the beginning of the file
> >>> f.readline()# Read the line, I get the expected 'bye\n'
> 'bye\n'
>
> It seems pretty clear to me that this is wrong.  If there is any
> caching going on, it should clearly be discarded if I do a seek.  Note
> that it's not just readline() that's returning me the wrong, cached
> data, as I've also tried this with read(), and I get the same
> results.  It's not acceptable that I have to close and reopen the file
> before every read when I'm doing random record access.
>
> So, is this a bug, or am I being stupid?

This has to do how the OS file-system operates. This is equivalent of
doing:

echo "hi" >foo.txt
vi foo.txt
in another session type: echo "bye" > foo.txt

the text in the vi session doesn't change.

you can even type 'rm foo.txt' and vi will still have the text there.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Possible read()/readline() bug?

[pruebauno]

On Oct 22, 3:44 pm, [EMAIL PROTECTED] wrote:
> On Oct 22, 2:54 pm, Mike Kent <[EMAIL PROTECTED]> wrote:
>
>
>
> > Before I file a bug report against Python 2.5.2, I want to run this by
> > the newsgroup to make sure I'm not being stupid.
>
> > I have a text file of fixed-length records I want to read in random
> > order.  That file is being changed in real-time by another process,
> > and my process want to see the changes to the file.  What I'm seeing
> > is that, once I've opened the file and read a record, all subsequent
> > seeks to and reads of that same record will return the same data as
> > the first read of the record, so long as I don't close and reopen the
> > file.  This indicates some sort of buffering and caching is going on.
>
> > Consider the following:
>
> > $ echo "hi" >foo.txt  # Create my test file
> > $ python2.5  # Run Python
> > Python 2.5.2 (r252:60911, Sep 22 2008, 16:13:07)
> > [GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2
> > Type "help", "copyright", "credits" or "license" for more information.
>
> > >>> f = open('foo.txt')  # Open my test file
> > >>> f.seek(0)# Seek to the beginning of the file
> > >>> f.readline() # Read the line, I get the data I expected
> > 'hi\n'
> > >>> # At this point, in another shell I execute 'echo "bye" >foo.txt'.  
> > >>> 'foo.txt' now has been changed
> > >>> # on the disk, and now contains 'bye\n'.
> > >>> f.seek(0)# Seek to the beginning of the still-open file
> > >>> f.readline() # Read the line, I don't get 'bye\n', I get 
> > >>> the original data, which is no longer there.
> > 'hi\n'
> > >>> f.close() # Now I close the file...
> > >>> f = open('foo.txt') # ... and reopen it
> > >>> f.seek(0)   # Seek to the beginning of the file
> > >>> f.readline()# Read the line, I get the expected 'bye\n'
> > 'bye\n'
>
> > It seems pretty clear to me that this is wrong.  If there is any
> > caching going on, it should clearly be discarded if I do a seek.  Note
> > that it's not just readline() that's returning me the wrong, cached
> > data, as I've also tried this with read(), and I get the same
> > results.  It's not acceptable that I have to close and reopen the file
> > before every read when I'm doing random record access.
>
> > So, is this a bug, or am I being stupid?
>
> This has to do how the OS file-system operates. This is equivalent of
> doing:
>
> echo "hi" >foo.txt
> vi foo.txt
> in another session type: echo "bye" > foo.txt
>
> the text in the vi session doesn't change.
>
> you can even type 'rm foo.txt' and vi will still have the text there.

Actually disregard what I said. vi loads everything in memory. You
might want to try:

f = open('foo.txt','r',0)

and see if that fixes your problem.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Possible read()/readline() bug?

[pruebauno]

On Oct 23, 9:48 am, Mike Kent <[EMAIL PROTECTED]> wrote:
> To followup on this:
>
> Terry: Yes, I did in fact miss the 'buffer' parameter to open.
> Setting the buffer parameter to 0 did in fact fix the test code that I
> gave above, but oddly, did not fix my actual production code; it
> continues to get the data as first read, rather than what is currently
> on the disk.  I'm still investigating why.
>
> Carl: I tried the above test code, without 'buffer=0' in the open, but
> with a flush added before reads in the appropriate places. The flush
> made no difference; readline continued to return the old data rather
> than what was actually on the disk.  So, flush isn't the answer.  I
> suppose that means that, when the document states it flushes the
> buffer, it's referring to the output buffer, not the input buffer.

Something odd is going on for sure. I had a couple of theories but
then I tested it on both Windows XP and AIX and could not reproduce
the problem even using the default buffer setting. As soon as I do a
seek and read it gives me the new data. I wonder if other people can
test this out on different operating systems and file systems and
detect a pattern.
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[pruebauno]

On Nov 12, 1:22 pm, Joe Strout <[EMAIL PROTECTED]> wrote:
> On Nov 12, 2008, at 10:45 AM, Tim Rowe wrote:
>
> > What do you actually mean by "Quacks like a string"? Supports the
> > 'count()' method? Then you find out if it doesn't when you try to
> > apply the 'count()' method. Supports some method that you don't
> > actually use? Then why do you care?
>
> Because if I write a method with the intention of treating the
> arguments like strings in various ways (slicing, combining with other
> strings, printing to stdout or writing to a file, etc. etc.), and some
> idiot (i.e. me six weeks later or long after I should have gone to
> bed) manages to accidentally pass in something else, then I want my
> program to blow up right away, not plant a roadside bomb and
> cheerfully wait for me to drive by.
>
> This is not hypothetical -- just last week I had a hard-to-track-down
> abend that ultimately turned out to be an NLTK.Tree object stored
> someplace that I expected to only contain strings.  I found it by
> littering my code with assertions of the form
> isinstance(foo,basestring).  If I'd had those in there in the first
> place, not only documenting my assumptions but letting the computer
> check them for me, it would have saved me a lot of grief.
>
> But in the spirit of duck-typing, I shouldn't actually check that foo
> is a basestring.  I should instead check that foo quacks like a
> basestring.  I'd define that is:
>
> "x quacks like a basestring if it implements all the public methods of
> basestring, and can be used in pretty much any context that a
> basestring can."
>
> I have to say "pretty much" since obviously there may be some evil
> context that actually checks isinstance.  But that's the pathological
> case, and we shouldn't let it prevent us from neatly handling the
> typical case.
>
> > The point about duck typing is that something might quack like a duck
> > but not walk like a duck -- one of those duck calls that hunters use,
> > for instance. Quacking like a duck doesn't actually mean it /is/ a
> > duck, it means that it will do instead of a duck if the quack is all
> > you want.
>
> Well, that's one point, but it's not the only point.  If I have code
> that expects to be working with strings, and I want to purposely give
> it something else, then it's reasonable to expect that the something-
> else will act like a string in every way that a string is likely to be
> exercised.  My string wrapper or doppleganger_string or whatever
> should implement all the methods, and support all the operators and
> type conversions, that basestring does.
>
> > If you need to know that it walks like a duck, mates like a duck and
> > tastes like a duck when roasted, you probably want it to really /be/ a
> > duck and should go back to inheritance.
>
> I can't agree; there are times when inheritance just won't do, for
> example when you don't have control over the object creation, because
> they come from some factory method you can't change.  In that case you
> may need to make a wrapper instead of a subclass, but if you've
> faithfully implemented the interface of the original class, you should
> be able to use it wherever the original class could be used (within
> reason).
>
> So, since it's pretty clear by now that there's no standard idiom for
> this, I'll try to cook up something myself.  For classes, I think we
> could do a two-stage test:
>
> 1. If the given object isinstance of the specified class, then all is
> good and return immediately.
> 2. Otherwise, check each of the public attributes of the specified
> class, and make sure that the given object has corresponding callable
> attributes.
>
> For case 2, we might be able to cache the result so that we don't do
> all that work again the next time the same type comparison is done.
>
> Anyway, I'll evolve something in our shop here and live with it a
> while, and in a few months I'll either share what we develop for this
> purpose, or admit it was a horrible idea all along.  :)
>
> Cheers,
> - Joe

It seems to me that what you are describing is exactly what abcs were
added for in 2.6, in particular registration:

class AnotherClass(metaclass=ABCMeta):
pass
AnotherClass.register(basestring)

assert isinstance(str, AnotherClass)

Please read this first:
http://www.python.org/dev/peps/pep-3119/

and tell us why that would not work.
--
http://mail.python.org/mailman/listinfo/python-list

Re: duck-type-checking?

[pruebauno]

On Nov 14, 12:47 am, George Sakkis <[EMAIL PROTECTED]> wrote:
> On Nov 13, 10:55 pm, Steven D'Aprano <[EMAIL PROTECTED]
>
> cybersource.com.au> wrote:
> > Take this example:
>
> > def foo(alist):
> > alist.sort()
> > alist.append(5)
>
> > The argument can be any object with sort and append methods (assumed to
> > act in place). But what happens if you pass it an object with a sort
> > method but no append? The exception doesn't occur until *after* the
> > object is sorted, which leaves it in an inconsistent state. This can be
> > undesirable: you might need the function foo to be atomic, either the
> > entire function succeeds, or none of it.
>
> In this example atomicity is not guaranteed even if alist is a builtin
> list (if it contains a complex number or other unorderable object),
> let alone if not isistance(alist, list). It gets worse: false
> positives are less likely for full-spelled methods with well-known
> names such as "sort" and "append"  (i.e. if hasattr(alist, 'append'),
> alist.append *probably* does what you think it does), but good luck
> with that when testing for __getitem__, __iter__ for more than one
> pass, __call__, and other special methods with ambiguous or undefined
> semantics.
>
> Let's face it, duck typing is great for small to medium complexity
> projects but it doesn't scale without additional support in the form
> of ABCs/interfaces, explicit type checking (at compile and/or run
> time), design by contract, etc. It must not be a coincidence that both
> Zope and Twisted had to come up with interfaces to manage their
> massive (for Python at least) complexity.
>
> George

What would be actually interesting would be an switch to the python
interpreter that internally annotated function parameters with how
they are used in the function and raised an exception as soon as the
function is called instead of later. Failing earlier rather than
later. Example:


def sub(x,y):
...run some stuff
...print x[2]
...return y.strip().replace('a','b')

internally python generates:

def sub(x: must have getitem, y: must have strip and replace)



sub([1,2,3,4],5)
Error calling sub(x,y): y has to have strip() method.

--
http://mail.python.org/mailman/listinfo/python-list

Re: redirecting stdout/err to mysql table

[pruebauno]

On Nov 18, 2:07 pm, n00b <[EMAIL PROTECTED]> wrote:
> greetings,
>
> i need to log to the db directly and wrote a little script to do so.
> since i'm pretty new to python,
> i was wondering if a) you could review the enclosed code and b)
> provide suggestions to harden to code to turn it into a more general,
> broadly reusable class.
>
> thank you very much.
>
> import sys
> import MySQLdb
>
> class DBLogger(object):
> def __init__(self):
> self.db_name = 'xxx'
> self.db_host = '127.0.0.1'
> self.db_table = 'yyy'
> self.db_uname = 'root'
> self.db_passwd = ''
> self.db_port = 3306
> self.db = None
> self.cur = None
> self.sql = 'INSERT INTO %s' %self.db_table + ' VALUES(null, NOW
> (), %s)'
>
> def openDb(self):
> try:
> self.db = MySQLdb.connect(host = self.db_host,
> user = self.db_uname,
> passwd = self.db_passwd,
> db = self.db_name,
> )
>
> self.cur = self.db.cursor()
> return self.db, self.cur
> except Exception, e:
> sys.stdout = sys.__stdout__
> sys.stderr = sys.__stderr__
> print e[0], e[1]
> sys.exit(1)
>
> def closeDb(self):
>self.cur.close()
>self.db.close()
>
> def write(self, string):
> s = string.strip('\n')
> if not s=='':
> self.openDb()
> self.cur.execute(self.sql, (s))
> self.db.commit()
> self.closeDb()
>
> dbl = DBLogger()
> sys.stdout = dbl
> sys.stderr = dbl
> #a = 'test string'
> #dbl.write(a)
>
> print 'a b c '
> sys.stdout = sys.__stdout__
> sys.stderr = sys.__stderr__
>
> thanks again for your time

Looks good to me.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Dynamic features used

[pruebauno]

On Nov 21, 4:17 am, [EMAIL PROTECTED] wrote:
> What are the dynamic features of Python that you use in your code?

The main ones is using configuration files that are plain Python
instead of XML and not having to wait 5 minutes to compile larger
programs. I also prefer structural typing over nominative typing and
duck typing is closer to structural than nominative typing with the
difference that compilation doesn’t take forever but the disadvantage
that the checking doesn’t happen until runtime. Hopefully they will
keep improving pylint to generate warnings for all those.

As far as speed goes, there might be 5% of my code that I would be
willing to run through a slow compiler to speed it up. For the rest it
is probably not worth it.
--
http://mail.python.org/mailman/listinfo/python-list

Re: starting a Python 2.5 executable without the DOS window

[pruebauno]

On Feb 27, 9:12 am, Greg Miller  wrote:
> I am working on a program that controls a piece of equipment.  The GUI/
> control software is written with Python2.5/wxPython.  I would like to
> know if there is a way of starting the GUI without the DOS window
> having to launch?  I would like only the application itself to appear,
> no DOS window.  Thanks for any information.
> Greg Miller

change the extension of your python program from ".py" to ".pyw".
Alternatively make sure that your program gets started by pythonw.exe
(insted of python.exe).
--
http://mail.python.org/mailman/listinfo/python-list

Re: Perl python - regex performance comparison

[pruebauno]

On Mar 3, 12:38 pm, Ivan  wrote:
> Hello everyone,
>
> I know this is not a direct python question, forgive me for that, but
> maybe some of you will still be able to help me. I've been told that
> for my application it would be best to learn a scripting language, so
> I looked around and found perl and python to be the nice. Their syntax
> and "way" is not similar, though.
> So, I was wondering, could any of you please elaborate on the
> following, as to ease my dilemma:
>
> 1. Although it is all relatively similar, there are differences
> between regexes of these two. Which do you believe is the more
> powerful variant (maybe an example) ?
>
> 2. They are both interpreted languages, and I can't really be sure how
> they measure in speed. In your opinion, for handling large files,
> which is better ?
> (I'm processing files of numerical data of several hundred mb - let's
> say 200mb - how would python handle file of such size ? As compared to
> perl ?)
>
> 3. This last one is somewhat subjective, but what do you think, in the
> future, which will be more useful. Which, in your (humble) opinion
> "has a future" ?
>
> Thank you for all the info you can spare, and expecially grateful for
> the time in doing so.
> -- Ivan
>
> p.s. Having some trouble posting this. If you see it come out several
> times, please ignore the copies.

1. They are so similar that, unless they added a lot in Perl lately,
both are about equally powerful. I think the difference had mainly to
do with how certain multiline constructs were handled. But that
doesn't make one more powerful than the other, it just means that you
have to make minor changes to port regexes from one to the other.

2. Speed is pretty similar too, Perl probably has a more optimized
regex engine because regexes are usually used a more in Perl programs.
Python probably has more optimized function calls, string methods,
etc. So it probably depends on the mix of functionality you are using.

3. Both languages have been around for a while and too much
infrastructure depends on them that they are not going to go away.
Perl is extensively used for sysadmin tasks, nagios, etc. and Python
is used in Gentoo, Redhat distributions. Of course COBOL is not dying
quickly either for the same reason which isn't the same than wanting
to program in it.

On this list you will find quite a few that have switched from Perl to
Python because we like the later better, but there are many
programmers that are quite happy with Perl. I myself want to play
around with Perl 6 at some point just for fun. Since a lot of the
reasons to choose one or another seem to be mostly syntax, I would
recommend you to write a couple of short programs in both of them and
see what you like more and use that.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Candidate for a new itertool

[pruebauno]

On Mar 7, 8:47 pm, Raymond Hettinger  wrote:
> The existing groupby() itertool works great when every element in a
> group has the same key, but it is not so handy when groups are
> determined by boundary conditions.
>
> For edge-triggered events, we need to convert a boundary-event
> predicate to groupby-style key function.  The code below encapsulates
> that process in a new itertool called split_on().
>
> Would love you guys to experiment with it for a bit and confirm that
> you find it useful.  Suggestions are welcome.
>
> Raymond
>
> -
>
> from itertools import groupby
>
> def split_on(iterable, event, start=True):
>     'Split iterable on event boundaries (either start events or stop
> events).'
>     # split_on('X1X23X456X', 'X'.__eq__, True)  --> X1 X23 X456 X
>     # split_on('X1X23X456X', 'X'.__eq__, False) --> X 1X 23X 456X
>     def transition_counter(x, start=start, cnt=[0]):
>         before = cnt[0]
>         if event(x):
>             cnt[0] += 1
>         after = cnt[0]
>         return after if start else before
>     return (g for k, g in groupby(iterable, transition_counter))
>
> if __name__ == '__main__':
>     for start in True, False:
>         for g in split_on('X1X23X456X', 'X'.__eq__, start):
>             print list(g)
>         print
>
>     from pprint import pprint
>     boundary = '--===2615450625767277916==\n'
>     email = open('email.txt')
>     for mime_section in split_on(email, boundary.__eq__):
>         pprint(list(mime_section, 1, None))
>         print '= = ' * 30

Sorry to hijack the thread but I now that you have a knack for finding
good iterator patterns. I have noticed a pattern lately: Aggregation
using a defaultdict. I quickly found two examples of problems that
could use this:

http://groups.google.com/group/comp.lang.python/browse_frm/thread/c8b3976ec3ceadfd

http://www.willmcgugan.com/blog/tech/2009/1/17/python-coder-test/

To show an example, using data like this:

>>> data=[('red',2,'other data'),('blue',5,'more data'),('yellow',3,'lots of 
>>> things'),('blue',1,'data'),('red',2,'random data')]

Then
>>> from itertools import groupby
>>> from operator import itemgetter
>>> from collections import defaultdict

We can use groupby to do this:
>>> [(el[0],sum(x[1] for x in el[1])) for el in 
>>> groupby(sorted(data,key=itemgetter(0)),itemgetter(0))]
[('blue', 6), ('red', 4), ('yellow', 3)]

>>> [(el[0],[x[1] for x in el[1]]) for el in 
>>> groupby(sorted(data,key=itemgetter(0)),itemgetter(0))]
[('blue', [5, 1]), ('red', [2, 2]), ('yellow', [3])]

>>> [(el[0],set([x[1] for x in el[1]])) for el in 
>>> groupby(sorted(data,key=itemgetter(0)),itemgetter(0))]
[('blue', set([1, 5])), ('red', set([2])), ('yellow', set([3]))]

But this way seems to be more efficient:

>>> def aggrsum(data,key,agrcol):
dd=defaultdict(int)
for el in data:
dd[key(el)]+=agrcol(el)
return dd.items()

>>> aggrsum(data,itemgetter(0),itemgetter(1))
[('blue', 6), ('yellow', 3), ('red', 4)]


>>> def aggrlist(data,key,agrcol):
dd=defaultdict(list)
for el in data:
dd[key(el)].append(agrcol(el))
return dd.items()

>>> aggrlist(data,itemgetter(0),itemgetter(1))
[('blue', [5, 1]), ('yellow', [3]), ('red', [2, 2])]


>>> def aggrset(data,key,agrcol):
dd=defaultdict(set)
for el in data:
dd[key(el)].add(agrcol(el))
return dd.items()

>>> aggrset(data,itemgetter(0),itemgetter(1))
[('blue', set([1, 5])), ('yellow', set([3])), ('red', set([2]))]


The data often contains objects with attributes instead of tuples, and
I expect the new namedtuple datatype to be used also as elements of
the list to be processed.

But I haven't found a nice generalized way for that kind of pattern
that aggregates from a list of one datatype to a list of key plus
output datatype that would make it practical and suitable for
inclusion in the standard library.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Mapping 64 bit int from C to Python-2.2

[pruebauno]

On Mar 9, 11:51 am, Explore_Imagination 
wrote:
> Hi
>
> I want to map 64 bit integers from C to python. I must use Python 2.2
> BUT There is no support for 64 bits integers in Python2.2 (Supported
> in 2.5).
>
> Now the problem is that I have these four variables:
>
> unit32_t a,b,c;
> uint64_t w,x,y,z;
>
> I use this funtion to map values:
>
> Py_BuildValue( "(lll)", a,b,c,w,x,y,z );
>
> As I access 32 bit values in Python it works fine BUT 64 bit intergers
> in Pythong give garbage values . I think there may be a case of
> overflow when 64 bit values in C are mapped to python.
>
> Any Suggestions?

You should be able to create PyLong objects from your 64 bit integers.
PyLong objects have unlimited integer precision. Alas I don't know
enough about the internal C code of Python to know what function to
use to do the conversion from an int64 to a PyLong object.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Candidate for a new itertool

[pruebauno]

On Mar 9, 6:55 pm, Raymond Hettinger  wrote:
> [prueba]
>
> > The data often contains objects with attributes instead of tuples, and
> > I expect the new namedtuple datatype to be used also as elements of
> > the list to be processed.
>
> > But I haven't found a nice generalized way for that kind of pattern
> > that aggregates from a list of one datatype to a list of key plus
> > output datatype that would make it practical and suitable for
> > inclusion in the standard library.
>
> Looks like you've searched the possibilities thoroughly and no one
> aggregation function seems to meet all needs.  That's usually a cue
> to not try to build one and instead let simple python loops do the
> work for you (that also saves the awkward itemgetter() calls in your
> examples).  To my eyes, all three examples look like straight-forward,
> easy-to-write, easy-to-read, fast plain python:
>
> >>> d = defaultdict(int)
> >>> for color, n, info in data:
> ...     d[color] += n
> >>> d.items()
>
> [('blue', 6), ('yellow', 3), ('red', 4)]
>
> >>> d = defaultdict(list)
> >>> for color, n, info in data:
>
> ...     d[color].append(n)>>> d.items()
>
> [('blue', [5, 1]), ('yellow', [3]), ('red', [2, 2])]
>
> >>> d = defaultdict(set)
> >>> for color, n, info in data:
> ...     d[color].add(n)
> >>> d.items()
>
> [('blue', set([1, 5])), ('yellow', set([3])), ('red', set([2]))]
>
> I don't think you can readily combine all three examples into a single
> aggregator without the obfuscation and awkwardness that comes from
> parameterizing all of the varying parts:
>
> def aggregator(default_factory, adder, iterable, keyfunc, valuefunc):
>    d = defaultdict(default_factory)
>    for record in iterable:
>        key = keyfunc(record)
>        value = valuefunc(record)
>        adder(d[key], value)
>    return d.items()
>
> >>> aggregator(list, list.append, data, itemgetter(0), itemgetter(1))
>
> [('blue', [5, 1]), ('yellow', [3]), ('red', [2, 2])]>>> aggregator(set, 
> set.add, data, itemgetter(0), itemgetter(1))
>
> [('blue', set([1, 5])), ('yellow', set([3])), ('red', set([2]))]
>
> Yuck!  Plain Python wins.
>
> Raymond
>
> P.S. The aggregator doesn't work so well for:
>
> >>> aggregator(int, operator.iadd, data, itemgetter(0), itemgetter(1))
>
> [('blue', 0), ('yellow', 0), ('red', 0)]
>
> The problem is that operator.iadd() doesn't have a way to both
> retrieve
> and store back into a dictionary.

Yes thinking about this more, one probably needs to have two code
paths depending if the type returned by default_factory is mutable or
immutable. But you are probably right that the ratio of redundancy/
variability is pretty low for such a function and the plain written
out for loop is not too painful. The only redundancy is the creation
and manipulation of the dictionary and the explicit looping.

--
http://mail.python.org/mailman/listinfo/python-list

Re: Rough draft: Proposed format specifier for a thousands separator

[pruebauno]

On Mar 12, 3:30 am, Raymond Hettinger  wrote:
> If anyone here is interested, here is a proposal I posted on the
> python-ideas list.
>
> The idea is to make numbering formatting a little easier with the new
> format() builtin
> in Py2.6 and Py3.0:  http://docs.python.org/library/string.html#formatspec
>
> -
>
> Motivation:
>
>     Provide a simple, non-locale aware way to format a number
>     with a thousands separator.
>
>     Adding thousands separators is one of the simplest ways to
>     improve the professional appearance and readability of
>     output exposed to end users.
>
>     In the finance world, output with commas is the norm.  Finance
> users
>     and non-professional programmers find the locale approach to be
>     frustrating, arcane and non-obvious.
>
>     It is not the goal to replace locale or to accommodate every
>     possible convention.  The goal is to make a common task easier
>     for many users.
>
> Research so far:
>
>     Scanning the web, I've found that thousands separators are
>     usually one of COMMA, PERIOD, SPACE, or UNDERSCORE.  The
>     COMMA is used when a PERIOD is the decimal separator.
>
>     James Knight observed that Indian/Pakistani numbering systems
>     group by hundreds.   Ben Finney noted that Chinese group by
>     ten-thousands.
>
>     Visual Basic and its brethren (like MS Excel) use a completely
>     different style and have ultra-flexible custom format specifiers
>     like: "_($* #,##0_)".
>
> Proposal I (from Nick Coghlan]:
>
>     A comma will be added to the format() specifier mini-language:
>
>     [[fill]align][sign][#][0][minimumwidth][,][.precision][type]
>
>     The ',' option indicates that commas should be included in the
> output as a
>     thousands separator. As with locales which do not use a period as
> the
>     decimal point, locales which use a different convention for digit
>     separation will need to use the locale module to obtain
> appropriate
>     formatting.
>
>     The proposal works well with floats, ints, and decimals.  It also
>     allows easy substitution for other separators.  For example:
>
>         format(n, "6,f").replace(",", "_")
>
>     This technique is completely general but it is awkward in the one
>     case where the commas and periods need to be swapped.
>
>         format(n, "6,f").replace(",", "X").replace(".", ",").replace
> ("X", ".")
>
> Proposal II (to meet Antoine Pitrou's request):
>
>     Make both the thousands separator and decimal separator user
> specifiable
>     but not locale aware.  For simplicity, limit the choices to a
> comma, period,
>     space, or underscore..
>
>     [[fill]align][sign][#][0][minimumwidth][T[tsep]][dsep precision]
> [type]
>
>     Examples:
>
>         format(1234, "8.1f")    -->     '  1234.0'
>         format(1234, "8,1f")    -->     '  1234,0'
>         format(1234, "8T.,1f")  -->     ' 1.234,0'
>         format(1234, "8T .f")   -->     ' 1 234,0'
>         format(1234, "8d")      -->     '    1234'
>         format(1234, "8T,d")      -->   '   1,234'
>
>     This proposal meets mosts needs (except for people wanting
> grouping
>     for hundreds or ten-thousands), but it comes at the expense of
>     being a little more complicated to learn and remember.  Also, it
> makes it
>     more challenging to write custom __format__ methods that follow
> the
>     format specification mini-language.
>
>     For the locale module, just the "T" is necessary in a formatting
> string
>     since the tool already has procedures for figuring out the actual
>     separators from the local context.
>
> Comments and suggestions are welcome but I draw the line at supporting
> Mayan numbering conventions ;-)
>
> Raymond

As far as I am concerned the most simple version plus a way to swap
around commas and period is all that is needed. The rest can be done
using one replace (because the decimal separator is always one of two
options). This should cover everywhere but the far east. 80% of cases
for 20% of implementation complexity.

For example:

[[fill]align][sign][#][0][,|.][minimumwidth][.precision][type]

> format(1234, ".8.1f")  --> ' 1.234,0'
> format(1234, ",8.1f")  --> ' 1,234.0'

--
http://mail.python.org/mailman/listinfo/python-list

Re: Candidate for a new itertool

[pruebauno]

On Mar 7, 8:47 pm, Raymond Hettinger  wrote:
> The existing groupby() itertool works great when every element in a
> group has the same key, but it is not so handy when groups are
> determined by boundary conditions.
>
> For edge-triggered events, we need to convert a boundary-event
> predicate to groupby-style key function.  The code below encapsulates
> that process in a new itertool called split_on().
>
> Would love you guys to experiment with it for a bit and confirm that
> you find it useful.  Suggestions are welcome.
>
> Raymond
>
> -
>
> from itertools import groupby
>
> def split_on(iterable, event, start=True):
>     'Split iterable on event boundaries (either start events or stop
> events).'
>     # split_on('X1X23X456X', 'X'.__eq__, True)  --> X1 X23 X456 X
>     # split_on('X1X23X456X', 'X'.__eq__, False) --> X 1X 23X 456X
>     def transition_counter(x, start=start, cnt=[0]):
>         before = cnt[0]
>         if event(x):
>             cnt[0] += 1
>         after = cnt[0]
>         return after if start else before
>     return (g for k, g in groupby(iterable, transition_counter))
>
> if __name__ == '__main__':
>     for start in True, False:
>         for g in split_on('X1X23X456X', 'X'.__eq__, start):
>             print list(g)
>         print
>
>     from pprint import pprint
>     boundary = '--===2615450625767277916==\n'
>     email = open('email.txt')
>     for mime_section in split_on(email, boundary.__eq__):
>         pprint(list(mime_section, 1, None))
>         print '= = ' * 30

For me your examples don't justify why you would need such a general
algorithm. A split function that works on iterables instead of just
strings seems straightforward, so maybe we should have that and
another one function with examples of problems where a plain split
does not work.
Something like this should work for the two examples you gave were the
boundaries are a known constants (and therefore there is really no
need to keep them. I can always add them later):

def split_on(iterable, boundary):
l=[]
for el in iterable:
if el!=boundary:
l.append(el)
else:
yield l
l=[]
yield l

def join_on(iterable, boundary):
it=iter(iterable)
firstel=it.next()
for el in it:
yield boundary
for x in el:
yield x

if __name__ == '__main__':
lst=[]
for g in split_on('X1X23X456X', 'X'):
print list(g)
lst.append(g)
print
print list(join_on(lst,'X'))
--
http://mail.python.org/mailman/listinfo/python-list

Re: Rough draft: Proposed format specifier for a thousands separator

[pruebauno]

On Mar 13, 7:06 am, Tim Rowe  wrote:
> 2009/3/12 Raymond Hettinger :
>
> > If anyone here is interested, here is a proposal I posted on the
> > python-ideas list.
>
> > The idea is to make numbering formatting a little easier with the new
> > format() builtin
> > in Py2.6 and Py3.0:  http://docs.python.org/library/string.html#formatspec
>
> As far as I can see you're proposing an amendment to *encourage*
> writing code that is not locale aware, with the amendment itself being
> locale specific, which surely has to be a regressive move in the 21st
> century. Frankly, I'd sooner see it made /harder/ to write code that
> is not locale aware (warnings, like FxCop gives on .net code?) tnan
> /easier/. Perhaps that's because I'm British, not American and I'm
> sick of having date fields get the date wrong because the programmer
> thinks the USA is the world. It makes me sympathetic to the problems
> caused to others by programmers who think the English-speaking world
> is the world.
>
> By the way, to others who think that 123,456.7 and 123.456,7 are the
> only conventions in common use in the West, no they're not. 123 456.7
> is in common use in engineering, at least in Europe, precisely to
> reduce (though not eliminate) problems caused by dot and comma
> confusion..
>
> --
> Tim Rowe

I lived in three different countries and in school used blank for
thousand separator to avoid confusion with the multiply operator. I
think this proposal is more for debugging big numbers and meant mostly
for programmers' eyes. We are already using the dot instead of comma
decimal separator in our programming languages that one more
Americanism won't kill us.

I am leaning towards proposal 1 now just to avoid the thousand
variations that will be requested because of this, making the
implementation unnecessarily complex. I can always use the 3
replacement hack (conveniently documented in the pep).

+1 for Nick's proposal
--
http://mail.python.org/mailman/listinfo/python-list

Re: How complex is complex?

[pruebauno]

On Mar 19, 1:25 pm, Paul Hildebrandt 
wrote:
> On Mar 19, 9:41 am, Kottiyath  wrote:
>
>
>
> > On Mar 19, 9:33 pm, Kottiyath  wrote:
>
> > > On Mar 19, 8:42 pm, Paul McGuire  wrote:
>
> > > > On Mar 19, 4:39 am, Kottiyath  wrote:
>
> > > > > I understand that my question was foolish, even for a newbie.
> > > > > I will not ask any more such questions in the future.
>
> > > > Gaaah! Your question was just fine, a good question on coding style.
> > > > I wish more people would ask such questions so that bad habits could
> > > > be avoided.
>
> > > > The newbie posts that are annoying are the ones that:
> > > > - are answered on page 1 of any tutorial ("how do I get the second
> > > > character of a string?")
> > > > - are obvious homework assignments with no actual effort on the
> > > > poster's part ("how do I write a Python program to find the first 10
> > > > prime numbers?")
> > > > - pontificate on what is wrong with Python, based on 2 hours'
> > > > experience with the language (often titled "What's wrong with Python",
> > > > with content like "Python sucks because it doesn't have a switch
> > > > statement/has significant whitespace/doesn't check types of arguments/
> > > > isn't totally object-oriented like Java/doesn't have interfaces/...")
> > > > - are so vague as to be just Usenet noise (titled "Help me", with no
> > > > content, or "i need to write a program and don't know where to start
> > > > can someone write it for me?")
>
> > > > I think Daniel's joke was on the rest of us, who each had to chime in
> > > > with our favorite dict processing algorithm.
>
> > > > It *would* be good for you as a newbie to get an appreciation of the
> > > > topics that were covered in these responses, though, especially the
> > > > distinction between updating the dict in-place vs. creating a new
> > > > dict.
>
> > > > -- Paul
>
> > > Daniel, Sorry for misunderstanding your post. I hope I was not being
> > > passive-aggresive - (also because I found that the second mechanism I
> > > provided was quite horrible :-), so I was indeed being foolish
> > > there. )
>
> > > Paul/Aahz, I did understand 2 things
> > > (1) When using map always consider that the function will be called
> > > everytime, so the hit on the performance is more.
> > > (2) The second mechanism and the first mechanism provides different
> > > solutions (new dict/same dict)
> > > both of which I did not think about at all.
>
> > > Also, thank you everyone for all the help. I have been following this
> > > thread for the last 4 months (when I started with python) and I have
> > > learned a lot. The amount of help provided here is amazing.
>
> > > p.s. -> English is indeed not my first language :-)
>
> > Oops, Forgot to mention the biggest learning.
>
> > Readability is better than brevity -
>
> I rewrote your sentence to be more optimized.
>
> Readability > brevity
>
> ;-)
>
> > Thanks to Rhodri.
>
> > This was a question which was bugging me all the time. When I look at
> > code, I am always envious when I see the same code written in much
> > smaller number of lines. Now, I will force myself to ask the questions
> > Rhodri proposed (esp: does it look uglier part) before deciding
> > whether or not to go ahead with brevity.
>
>

sometimes: brevity==Readability

but as in many things the trick is in finding the right tradeoff. I am
willing to accept some trivial amount of additional complexity if it
means I have to read less lines of code, but I have my limits too. I
aim for a comprehension speed of 2-10 lines per minute for somebody
proficient in the language.
--
http://mail.python.org/mailman/listinfo/python-list

Re: improve this newbie code/nested functions in Python?

[pruebauno]

On Mar 19, 10:21 pm, Esmail  wrote:
> Hi,
>
> I'm new to writing Python code. This is a simple client I wrote, it
> works, but I feel it doesn't look as clean as it could. Can anyone
> make suggestions how to streamline this code?
>
> Also, I am using two nested functions, it seems that nested functions
> aren't used that much in Python - is that correct? And if so, how
> come?
>
> thanks,
>
> Esmail
>
> ps: I realize there is minimal error checking/exception handling.
>
> #!/usr/bin/env python
>
> import sys
> from socket import *
> from threading import Thread
>
> class Client(object):
>     def __init__(self, host="localhost", port=, name = "esmail"):
>         self._host = host
>         self._port = port
>         self._name = name
>         self._address=(self._host, self._port)
>         self._sock=socket(AF_INET, SOCK_STREAM)
>         self._sock.connect(self._address)
>         self._parent = self
>         self._keepGoing = True
>
>     def info(self):
>         return self._host, self._port, self._name
>
>     class Listener(Thread):
>        def __init__(self, parent, tname="listener"):
>            Thread.__init__(self,name = tname)
>            self._parent = parent
>            print self._parent._host
>
>        def run(self):
>
>            while self._parent._keepGoing:
>                m = self._parent._sock.recvfrom(1024)
>                print m[0]
>
>     class Speaker(Thread):
>        def __init__(self, parent, tname = "speaker"):
>            Thread.__init__(self,name = tname)
>            self._parent = parent
>            self._parent._sock.send(self._parent._name + "\n")
>
>        def run(self):
>
>            while(True):
>                m = raw_input("-> ")
>                if m == "bye":
>                    self._parent._sock.send(self._parent._name + " is
> signing off.\n")
>                    self._parent._sock.send("bye\n")
>                    self._parent._keepGoing = False
>                    break;
>                else:
>                    self._parent._sock.send(m + "\n")
>
> def main():
>
>     if len(sys.argv) == 4:  # prog name + 3 args
>         host = sys.argv[1]
>         port = int(sys.argv[2])
>         name = sys.argv[3]
>         c = Client(host, port, name)
>     else:
>         c = Client()
>
>     print "Client connecting to - host=%s  port=%d   name=%s" % c.info
> ()
>
>     s = Client.Speaker(c)
>     s.start()
>
>     l = Client.Listener(c)
>     l.start()
>
> main()

How about renaming the Client to SocketConfig, Listener to
ClientListener and Speaker to ClientSpeaker and put all at the same
level. The you can do this:

 c = SocketConfig(host, port, name)
 s = ClientSpeaker(c)
 l = ClientListener(c)

the other option would be to create a Speaker and Listener factory in
Client that returns Speakers and Listeners so you can do:

c = Client(host, port, name)
s = c.Speaker()
l = c.Listener()

--
http://mail.python.org/mailman/listinfo/python-list

Re: Unit testing frameworks

[pruebauno]

On Mar 24, 8:06 am, grkunt...@gmail.com wrote:
> I am looking for a unit testing framework for Python. I am aware of
> nose, but was wondering if there are any others that will
> automatically find and run all tests under a directory hierarchy.
>
> Thanks, Ralph

*Nose
*Trial
*py.test
--
http://mail.python.org/mailman/listinfo/python-list

Re: Another form of dynamic import

[pruebauno]

On Mar 25, 10:23 am, Marco Nawijn  wrote:
> Hello,
>
> In short I would like to know if somebody knows if it is possible to
> re-execute a statement that raised an exception? I will explain the
> reason by providing a small introduction on why this might be nice in
> my case
> and some example code.
>
> I am using the python bindings to a *very* large C++ library. About
> 5000 classes divided over approx. 450 different
> packages are exposed through the Python interface. To reduce the
> number of import statements that need to be inserted and to limit the
> number of wildcard imports it would be very helpful if class names
> could be automatically imported from the proper module. There is no
> problem in finding out the proper module given a (valid) class name.
>
> As an example, look at the following statement
>
> >> aPoint = gp_Pnt(1.0, 0.0, 0.0)     # Oops, this will raise a NameError, 
> >> since
>
>                                                       # gp_Pnt class
> is unknown
>
> NameError: name 'gp_Pnt' is not defined
>
> As indicated, this will raise a NameError exception. What I would like
> to do is something like the following (pseudo-code):
>
> try:
>     
>     
>     aPoint = gp_Pnt(1.0, 0.0, 0.0)    [1]
>
>     
>     
> except NameError, e:
>
>      name = e.args[0].split[1]
>
>      if isValid(name):
>           doImport(name)
> ===> Can I go back to statement [1] from this point?
>      else:
>          raise e
>
> There is no problem in catching the exception, finding out which name
> is unknown to python and check if this is a valid name for my library.
> My question is, is there any possibility of going back to the
> statement that raised the error, re-execute the statement and
> continue?
>
> Thanks for any thoughts and suggestions.
>
> Marco

You can always use a loop:

recover=True
while True:
try:


aPoint = gp_Pnt(1.0, 0.0, 0.0)[1]


except NameError, e:
if recover:
recover=False
name = e.args[0].split[1]
if isValid(name):
doImport(name)
else:
raise e
else:
 break


--
http://mail.python.org/mailman/listinfo/python-list

Re: Programming Python 4th Edition?

[pruebauno]

On Mar 26, 10:08 pm, Esmail  wrote:
> Hi,
>
> Does anyone know if there is a 4th edition of this book planned
> and if so, when it might be coming out?
>
> It looks like a good and comprehensive book but is getting a bit
> outdated(?).
>
> And I guess since I'm asking this, I might as well be asking what
> your favorite, comparable Python book might be :-)
>
> Thanks,
> Esmail

Make sure that Programming Python is what you want. I was very
disappointment with it (First edition) because I was expecting
something like the Perl Camel book and it isn't.

It isn't a introduction to the Python language like "Learning Python",
it doesn't work as reference like "Python in a Nutshell", it doesn't
contain short idiomatic code like "Python Cookbook". What you are left
with is different application domains and how to apply Python to them.
The book is excellent if you want to do Network, GUI, Databases, etc.
but poor if you want to learn about Python the core language. The
title of the book should be changed from "Programming Python" to
"Applied Python" so newcomers to the language don't buy it by mistake.
Even the preface says that it is about "application-level programming
in Python". The book is pretty much 4 books in 1:
  System programming in Python
  Tkinter programming in Python
  Internet programming in Python
  Database and Persistence programming in Python

In my opinion an experienced programmer that wants to learn Python the
language should buy "Nutshell" and somebody new to programming in
general "Learning". You should buy "Programming" only if you need to
develop in one of the Domains covered in the book.
--
http://mail.python.org/mailman/listinfo/python-list

Re: complaints about no replies last week

[pruebauno]

On Mar 28, 11:07 am, Aaron Brady  wrote:
> Hi,
>
> A week ago, I posted a question and an idea about Python's garbage
> collector.  I got a few replies.  Some days later, I posted a mock-up
> implementation of it, and got *NO* replies.  Does this mean:
>
> a) It works
> b) It doesn't work
> c) It's not particularly applicable to Python at that point
> (particularly)
> d) It's not news
>
> Thanks and sincerely.
>
> P.S.  Non-plugging 
> links:http://groups.google.com/group/comp.lang.python/browse_thread/thread/...http://groups.google.com/group/comp.lang.python/browse_thread/thread/...

e) It is a hard or complex problem that requires significant
investment of time on a problem or approach that few people are
interested in at the moment.

f) The description is confusing or incomplete or the source code is
long and difficult to read.

I myself asked about how to write a library to efficiently do union
and intersection of sets containing time intervals some time ago on
this list and got little to no answers. It is a tricky problem. Since
I was getting paid I got an O(n*n) solution working. People on this
list on the other hand do not get paid and answer whatever strikes
their fancy. Sometimes the question is hard or confusing and nobody is
motivated enough to answer.
--
http://mail.python.org/mailman/listinfo/python-list

Re: complaints about no replies last week

[pruebauno]

On Mar 31, 2:56 am, Arnaud Delobelle  wrote:
> Arnaud Delobelle wrote:
> > prueba...@latinmail.com writes:
> > [...]
> > > I myself asked about how to write a library to efficiently do union
> > > and intersection of sets containing time intervals some time ago on
> > > this list and got little to no answers. It is a tricky problem. Since
> > > I was getting paid I got an O(n*n) solution working. People on this
> > > list on the other hand do not get paid and answer whatever strikes
> > > their fancy. Sometimes the question is hard or confusing and nobody is
> > > motivated enough to answer.
>
> > I wasn't around when you posted this I guess. Do you mean intervals sets
> > on the (real) number line such as:
>
> >       1-3, 6-10 meaning all numbers between 1 and 3 and all numbers
> >       between 6 and 10.
>
> > In this case I think you can achieve union and intersection in O(nlogn)
> > where n is the total number of intervals in the interval sets to unify
> > or intersect. There is an implementation below. I have chosen a very
> > simple data structure for interval sets: an interval set is the list of
> > its endpoints. E.g.
>
> >     1-3, 6-10 is the list [1, 3, 6, 10]
>
> > This means that I can't specify whether an interval is closed or open.
> > So in the implementation below all intervals are assumed to be open.
> > The method could be made to work for any kind of intervals with the same
> > complexity, there would just be a few more LOC.  I'm focusing on the
> > principle - here it is:
>
> > --
> > # Implementation of union and intersection of interval sets.
>
> > from itertools import *
>
> > def combine(threshold, intsets):
> >     endpoints = sorted(chain(*imap(izip, intsets, repeat(cycle([1,-1])
> >     height = 0
> >     compound = []
> >     for x, step in endpoints:
> >         old_height = height
> >         height += step
> >         if max(height, old_height) == threshold:
> >             compound.append(x)
> >     return compound
>
> > def union(*intsets):
> >     return combine(1, intsets)
>
> > def intersection(*intsets):
> >     return combine(len(intsets), intsets)
>
> > # tests
>
> > def pretty(a):
> >     a = iter(a)
> >     return ', '.join("%s-%s" % (a, b) for a, b in izip(a, a))
>
> > tests = [
> >     ([1, 5, 10, 15], [3, 11, 13, 20]),
> >     ([2, 4, 6, 8], [4, 7, 10, 11]),
> >     ([0, 11], [5, 10, 15, 25], [7, 12, 13, 15]),
> >     ]
>
> > for intsets in tests:
> >     print "sets: ", "; ".join(imap(pretty, intsets))
> >     print "union: ", pretty(union(*intsets))
> >     print "intersection: ", pretty(intersection(*intsets))
> >     print "-"*20
> > --
>
> > Is this what you were looking for?
>
> > --
> > Arnaud
>
> I realised after posting last night that I must be
>
> (1) solving the wrong problem
> (2) solving it badly
>
> - My implementation of the combine() function above is O(nlogn)
> (because of the sorted() call) whereas it could be O(n) by iterating
> over the interval in the parallel manner, hence (2).  This would make
> union() and intersection() O(n).
>
> - As the problem was solved by the OP in O(n^2) I must be solving the
> wrong problem (1).
>
> I apologise for this.
>
> However it was a nice and compact implementation IMHO :)
>
> --
> Arnaud

I am pretty sure the problem can be solved in O(n log n). I just
wasn't feeling overly smart when I was writing the algorithm. N is on
average 4 and it had eventually to be implemented inside a framework
using C++ anyway, so it is pretty fast. I can’t believe that no
programmer has come over the same kind of problem before, yet my
Google fu didn’t do anything for me.

Well since I attracted a couple people's attention I will describe the
problem in more detail. Describing the problem properly is probably as
hard as solving it, so excuse me if I struggle a bit.

The problem is for a health insurance company and involves the period
of time a person is covered. Most insurance companies allow not only
for the main member to be insured but his family: the spouse and the
dependents (children). This additional coverage costs extra but less
than a full new insurance. So for example if Alice buys an insurance
worth at 100 dollars a month, she can insure her husband Bob for an
additional 50 dollars. Under certain circumstances Alice may go off
the insurance and only Bob stays. In that case the price goes back to
100 dollars or maybe there is a deal for 80 or something like that. In
other words the cost of the insurance is dependent on the combination
of family members that participate in it. Additionally not only do we
have different family compositions but also different insurance
products. So you can get medical, dental and vision insurance.

All that data is stored in a database that is not very tidy and looks
something like this:

First Day of Coverage, Last Day of Coverage, Relationship, Product
5/3/2005, 5/3/2005, D, M
9/10/

Re: complaints about no replies last week

[pruebauno]

On Mar 31, 4:07 pm, Arnaud Delobelle  wrote:
> prueba...@latinmail.com writes:
>
> [...]
>
>
>
> > Well since I attracted a couple people's attention I will describe the
> > problem in more detail. Describing the problem properly is probably as
> > hard as solving it, so excuse me if I struggle a bit.
>
> > The problem is for a health insurance company and involves the period
> > of time a person is covered. Most insurance companies allow not only
> > for the main member to be insured but his family: the spouse and the
> > dependents (children). This additional coverage costs extra but less
> > than a full new insurance. So for example if Alice buys an insurance
> > worth at 100 dollars a month, she can insure her husband Bob for an
> > additional 50 dollars. Under certain circumstances Alice may go off
> > the insurance and only Bob stays. In that case the price goes back to
> > 100 dollars or maybe there is a deal for 80 or something like that. In
> > other words the cost of the insurance is dependent on the combination
> > of family members that participate in it. Additionally not only do we
> > have different family compositions but also different insurance
> > products. So you can get medical, dental and vision insurance.
>
> > All that data is stored in a database that is not very tidy and looks
> > something like this:
>
> > First Day of Coverage, Last Day of Coverage, Relationship, Product
> > 5/3/2005, 5/3/2005, D, M
> > 9/10/2005, 10/10/2005, S, V
> > 3/15/2005, 7/15/2005, M, M
> > 3/1/2005, 6/1/2005, S, M
> > 5/15/2005, 7/20/2005, D, D
> > 9/10/2005, 1/1/2140, M, V
> > 2/1/2005, 5/3/2005, M, M
>
> > Where
> > Relationship: M=Main Member, S=Spouse, D=Dependent
> > Product: M=Medical, D=Dental, V=Vision
>
> > As far as I know at the present time there are no deals based on
> > combination of products purchased so we will handle each product
> > independently. What I want is a simple algorithm that allows me to
> > calculate something like this out of it (hopefully I didn’t make a
> > mistake):
>
> > Medical:
> > 2/1/2005, 2/28/2005, M
> > 3/1/2005, 5/2/2005, M&S
> > 5/3/2005, 5/3/2005, M&S&D
> > 5/4/2005, 6/1/2005, M&S
> > 6/2/2005, 7/15/2005, M
>
> > Dental:
> > 5/15/2005, 7/20/2005, D
>
> > Vision:
> > 9/10/2005, 10/10/2005, M&S
> > 10/11/2005, 1/1/2140, M
>
> OK the approach I describe in my previous email works fine for this
> particular problem. I implement it below - the function walk_ivals is at
> the heart of it, I've made it as simple as possible and it's pretty
> clear that it is O(nlogn).
>
> The function that actually sorts the data is union(), and it's just a
> call to walk_ivals with callback the function acc() which is constant
> time, therefore union() itself has the same complexity as walk_ivals.
>
> There are no comments - I don't have the time to add any, sorry!
>
> --
> import datetime
> from collections import defaultdict
>
> def walk_ivals(ivals, callback, endvals=(-1, 1)):
>     endpoints = [(x, data) for ival, data in ivals for x in zip(ival, 
> endvals)]
>     endpoints.sort()
>     for (endpoint, endval), data in endpoints:
>         callback(endpoint, endval, data)
>
> def union(ivals):
>     timelines = defaultdict(list)
>     mult = defaultdict(lambda: defaultdict(int))
>     def acc(date, step, data):
>         rel, prod = data
>         old_mult = mult[prod][rel]
>         mult[prod][rel] -= step
>         if not(old_mult and mult[prod][rel]):
>             rels = [rel for rel, m in mult[prod].iteritems() if m]
>             if timelines[prod] and timelines[prod][-1][0] == date:
>                 timelines[prod][-1][1] = rels
>             else:
>                 timelines[prod].append([date, rels])
>     walk_ivals(ivals, acc)
>     return timelines
>
> test_data = """5/3/2005, 5/3/2005, D, M
> 9/10/2005, 10/10/2005, S, V
> 3/15/2005, 7/15/2005, M, M
> 3/1/2005, 6/1/2005, S, M
> 5/15/2005, 7/20/2005, D, D
> 9/10/2005, 1/1/2140, M, V
> 2/1/2005, 5/3/2005, M, M"""
>
> def parse_date(date_string):
>     month, day, year = map(int, date_string.split('/'))
>     return datetime.date(year, month, day)
>
> def parse_data(data_string):
>     data = []
>     for line in data_string.split("\n"):
>         start, end, rel, prod = line.split(",")
>         start, end = map(parse_date, (start + datetime.timedelta(1), end))
>         rel, prod = rel.strip(), prod.strip()
>         data.append([(start, end), (rel, prod)])
>     return data
>
> def test():
>     ivals = parse_data(test_data)
>     timelines = union(ivals)
>     for prod, timeline in timelines.iteritems():
>         print "-"*20
>         print "Product", prod
>         for date, covers in timeline:
>             print date, ' & '.join(covers)
>
> if __name__ == '__main__':
>    test()      
> --
>
> Here is what it outputs:
>
> marigold:junk arno$ python intervals2.py
> 
> Product M
> 2005-02-01 M
> 2

Help improve program for parsing simple rules

[pruebauno]

Another interesting task for those that are looking for some
interesting problem:
I inherited some rule system that checks for programmers program
outputs that to be ported: given some simple rules and the values it
has to determine if the program is still working correctly and give
the details of what the values are. If you have a better idea of how
to do this kind of parsing please chime in. I am using tokenize but
that might be more complex than it needs to be. This is what I have
come up so far:

rules=[
 '( A - B ) = 0',
 '(A + B + C + D + E + F + G + H + I) = J',
 '(A + B + C + D + E + F + G + H) = I',
 '(A + B + C + D + E + F) = G',
 '(A + B + C + D + E) = (F + G + H + I + J)',
 '(A + B + C + D + E) = (F + G + H + I)',
 '(A + B + C + D + E) = F',
 '(A + B + C + D) = (E + F + G + H)',
 '(A + B + C) = (D + E + F)',
 '(A + B) = (C + D + E + F)',
 '(A + B) = (C + D)',
 '(A + B) = (C - D + E - F - G + H + I + J)',
 '(A + B) = C',
 '(A + B) = 0',
 '(A+B+C+D+E) = (F+G+H+I+J)',
 '(A+B+C+D) = (E+F+G+H)',
 '(A+B+C+D)=(E+F+G+H)',
 '(A+B+C)=(D+E+F)',
 '(A+B)=(C+D)',
 '(A+B)=C',
 '(A-B)=C',
 '(A/(B+C))',
 '(G + H) = I',
 '-0.99 LE ((A+B+C)-(D+E+F+G)) LE 0.99',
 '-0.99 LE (A-(B+C)) LE 0.99',
 '-1000.00 LE A LE 0.00',
 '-5000.00 LE A LE 0.00',
 'A < B',
 'A < 7000',
 'A = -(B)',
 'A = C',
 'A = 0',
 'A GT 0',
 'A GT 0.00',
 'A GT 7.00',
 'A LE B',
 'A LT -1000.00',
 'A LT -5000',
 'A LT 0',
 'A=(B+C+D)',
 'A=B',
 'I = (G + H)',
 '0.00 LE A LE 4.00',
 '4.00 LT A LE 7.00'
 ]
vars_={'A': 0, 'B': 1.1, 'C': 2.2, 'D': 3.3, 'E': 4.4, 'F': 5.5, 'G':
6.6, 'H':7.7, 'I':8.8, 'J':9.9}

import tokenize as TOK
import StringIO as SIO

COMP_REPLACERS={'LT':'<', 'GT':'>', 'LE':'<=', 'GE':'>=', '=':'==',
'=>':'=>', '=<':'=<'}

def get_tokens(string):
return [x[:2] for x in TOK.generate_tokens(SIO.StringIO
(string).readline)][:-1]

def replace_comps(toks,repl):
return [(TOK.OP, repl[x[1]]) if x[1] in repl else x for x in toks]

def replace_names(norm,vars_):
return [(TOK.NUMBER,str(vars_.get(x[1],x[1]))) if x[0]==TOK.NAME
else x for x in norm]

def split_seccions(subs,comp_split):
l=[]
g=[]
for type_,value in subs:
if value in comp_split:
g.append((l,value))
l=[]
else:
l.append((type_,value))
g.append((l,''))
return g

def all_seccions(grps):
return [(TOK.untokenize(lst),comper) for lst,comper in grps]

def calc_seccions(rep):
return [(str(eval(comp,{},{})),comper) for comp,comper in rep]

def calc_deltas(calc):
return [eval(calc[i][0]+'-'+calc[i+1][0],{},{}) for i in range(len
(calc)-1)]

def main():
for cur_rule in rules[20:26]:
tokens=get_tokens(cur_rule)
normal=replace_comps(tokens,COMP_REPLACERS)
subst=replace_names(normal,vars_)
groups=split_seccions(subst,COMP_REPLACERS.values())
rep=all_seccions(groups)
rep_out=''.join(x[0]+x[1] for x in rep)
calc=calc_seccions(rep)
calc_out=''.join(x[0]+x[1] for x in calc)
deltas=calc_deltas(calc)
result=eval(calc_out,{},{})
print
print 'Values:',', '.join(str(key)+': '+str(val) for key,val
in sorted(vars_.iteritems()))
print 'Read rule:   ',cur_rule
print 'Used rule:   ',TOK.untokenize(normal)
print 'Substitution:',rep_out
print 'Calculation: ',calc_out
print 'Result:  ','Successful' if result else 'Failed'
if  not result and '==' in calc_out:
print 'Difference:  ',', '.join(map(str,deltas))
print '='*40


if __name__=='__main__': main()
--
http://mail.python.org/mailman/listinfo/python-list

Re: Help improve program for parsing simple rules

[pruebauno]

On Apr 16, 3:59 pm, Aaron Brady  wrote:
> On Apr 16, 10:57 am, prueba...@latinmail.com wrote:
>
> > Another interesting task for those that are looking for some
> > interesting problem:
> > I inherited some rule system that checks for programmers program
> > outputs that to be ported: given some simple rules and the values it
> > has to determine if the program is still working correctly and give
> > the details of what the values are. If you have a better idea of how
> > to do this kind of parsing please chime in. I am using tokenize but
> > that might be more complex than it needs to be. This is what I have
> > come up so far:
>
> > rules=[
> >          '( A - B ) = 0',
> >          '(A + B + C + D + E + F + G + H + I) = J',
> >          '(A + B + C + D + E + F + G + H) = I',
> >          '(A + B + C + D + E + F) = G',
> >          '(A + B + C + D + E) = (F + G + H + I + J)',
> >          '(A + B + C + D + E) = (F + G + H + I)',
> >          '(A + B + C + D + E) = F',
> >          '(A + B + C + D) = (E + F + G + H)',
> >          '(A + B + C) = (D + E + F)',
> >          '(A + B) = (C + D + E + F)',
> >          '(A + B) = (C + D)',
> >          '(A + B) = (C - D + E - F - G + H + I + J)',
> >          '(A + B) = C',
> >          '(A + B) = 0',
> >          '(A+B+C+D+E) = (F+G+H+I+J)',
> >          '(A+B+C+D) = (E+F+G+H)',
> >          '(A+B+C+D)=(E+F+G+H)',
> >          '(A+B+C)=(D+E+F)',
> >          '(A+B)=(C+D)',
> >          '(A+B)=C',
> >          '(A-B)=C',
> >          '(A/(B+C))',
> >          '(G + H) = I',
> >          '-0.99 LE ((A+B+C)-(D+E+F+G)) LE 0.99',
> >          '-0.99 LE (A-(B+C)) LE 0.99',
> >          '-1000.00 LE A LE 0.00',
> snip
> > def main():
> >     for cur_rule in rules[20:26]:
> >         tokens=get_tokens(cur_rule)
> >         normal=replace_comps(tokens,COMP_REPLACERS)
> >         subst=replace_names(normal,vars_)
> >         groups=split_seccions(subst,COMP_REPLACERS.values())
> >         rep=all_seccions(groups)
> >         rep_out=''.join(x[0]+x[1] for x in rep)
> >         calc=calc_seccions(rep)
> >         calc_out=''.join(x[0]+x[1] for x in calc)
> >         deltas=calc_deltas(calc)
> >         result=eval(calc_out,{},{})
>
> snip
>
> You are using 'eval' which isn't safe if its inputs are dangerous.  If
> you are controlling the inputs, you might be interested in the
> optional arguments to 'eval'.
>
> >>> a= '-1000.00 < A < 0.00'
> >>> eval( a, { 'A': -100 } )
> True
> >>> eval( a, { 'A': -1000 } )
>
> False
>
> The syntax is slightly different for Python 2.  For the replacement of
> 'LE', I assume you require spaces on both sides.
>
> >>> a= '-1000.00 LE A LE 0.00'
> >>> b= a.replace( ' LE ', ' <= ' )
> >>> b
>
> '-1000.00 <= A <= 0.00'>>> eval( b, { 'A': -1000 } )
> True
> >>> eval( b, { 'A': -1001 } )
>
> False
>
> If you need something more flexible, the 'ast' module gives you more
> options.  The string has to be a valid Python module to start with.
>
> FYI, have you checked order of precedence in your custom rule set to
> match Python's?

I know about evals implication of safety. Rules are defined by the
programmers so I don't worry too much about it at this point. They
should know better than messing up their application server. Unless
there is some easier way to do it I am willing to take the risk.
Precedence is standard math, we can always add some extra parenthesis
to the rules, I don't thing the old system would mind.

I thought about using eval with a locals dictionary, but they want
output of the intermediate values. I want to avoid the situation where
the intermediate output does not match what eval is doing.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Help improve program for parsing simple rules

[pruebauno]

On Apr 16, 9:29 pm, John Machin  wrote:
> On Apr 17, 1:57 am, prueba...@latinmail.com wrote:
>
> > COMP_REPLACERS={'LT':'<', 'GT':'>', 'LE':'<=', 'GE':'>=', '=':'==',
> > '=>':'=>', '=<':'=<'}
>
> What do the '=>' and '=<' represent? Why are you replacing each by
> itself?

because of this:
groups=split_seccions(subst,COMP_REPLACERS.values())

I didn't want to create a separate variable. Those are dubious anyway,
I haven't seen one in all of our current rules and it generates a
syntax error in Python so I took them out:

COMP_REPLACERS={'LT':'<', 'GT':'>', 'LE':'<=', 'GE':'>=', '=':'=='}

And before somebody else points it out. This line:

(TOK.NUMBER,str(vars_.get(x[1],x[1]))) if x[0]==TOK.NAME

should really be:

(TOK.NUMBER,str(vars_[x[1]])) if (x[0]==TOK.NAME and x[1] in vars_)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Help improve program for parsing simple rules

[pruebauno]

On Apr 17, 11:26 am, Paul McGuire  wrote:
> On Apr 16, 10:57 am, prueba...@latinmail.com wrote:
>
> > Another interesting task for those that are looking for some
> > interesting problem:
> > I inherited some rule system that checks for programmers program
> > outputs that to be ported: given some simple rules and the values it
> > has to determine if the program is still working correctly and give
> > the details of what the values are. If you have a better idea of how
> > to do this kind of parsing please chime in. I am using tokenize but
> > that might be more complex than it needs to be. This is what I have
> > come up so far:
>
> I've been meaning to expand on pyparsing's simpleArith.py example for
> a while, to include the evaluation of the parsed tokens.  Here is the
> online version,http://pyparsing.wikispaces.com/file/view/eval_arith.py,
> it will be included in version 1.5.2 (coming shortly).  I took the
> liberty of including your rule set as a list of embedded test cases.
>
> -- Paul

That is fine with me. I don't know how feasible it is for me to use
pyparsing for this project considering I don't have admin access on
the box that is eventually going to run this. To add insult to injury
Python is in the version 2->3 transition (I really would like to push
the admins to install 3.1 by the end of the year before the amount of
code written by us gets any bigger) meaning that any third party
library is an additional burden on the future upgrade. I can't
remember if pyparsing is pure Python. If it is I might be able to
include it alongside my code if it is not too big.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Help improve program for parsing simple rules

[pruebauno]

On Apr 17, 5:32 pm, Paul McGuire  wrote:
> On Apr 17, 2:40 pm, prueba...@latinmail.com wrote:
>
>
>
> > On Apr 17, 11:26 am, Paul McGuire  wrote:
>
> > > On Apr 16, 10:57 am, prueba...@latinmail.com wrote:
>
> > > > Another interesting task for those that are looking for some
> > > > interesting problem:
> > > > I inherited some rule system that checks for programmers program
> > > > outputs that to be ported: given some simple rules and the values it
> > > > has to determine if the program is still working correctly and give
> > > > the details of what the values are. If you have a better idea of how
> > > > to do this kind of parsing please chime in. I am using tokenize but
> > > > that might be more complex than it needs to be. This is what I have
> > > > come up so far:
>
> > > I've been meaning to expand on pyparsing's simpleArith.py example for
> > > a while, to include the evaluation of the parsed tokens.  Here is the
> > > online version,http://pyparsing.wikispaces.com/file/view/eval_arith.py,
> > > it will be included in version 1.5.2 (coming shortly).  I took the
> > > liberty of including your rule set as a list of embedded test cases.
>
> > > -- Paul
>
> > That is fine with me. I don't know how feasible it is for me to use
> > pyparsing for this project considering I don't have admin access on
> > the box that is eventually going to run this. To add insult to injury
> > Python is in the version 2->3 transition (I really would like to push
> > the admins to install 3.1 by the end of the year before the amount of
> > code written by us gets any bigger) meaning that any third party
> > library is an additional burden on the future upgrade. I can't
> > remember if pyparsing is pure Python. If it is I might be able to
> > include it alongside my code if it is not too big.- Hide quoted text -
>
> > - Show quoted text -
>
> It *is* pure Python, and consists of a single source file for the very
> purpose of ease-of-inclusion.  A number of projects include their own
> versions of pyparsing for version compatibility management, matplotlib
> is one that comes to mind.
>
> The upcoming version 1.5.2 download includes a pyparsing_py3.py file
> for Python 3 compatibility, I should have that ready for users to
> download *VERY SOON NOW*!
>
> -- Paul

Thanks,
I will consider it. I have to admit that although it looks like it is
a very good solution, it is also longer and more complex than my
current code. Having to explicitly define standard python evaluation
and semantics is a bit overkill.
--
http://mail.python.org/mailman/listinfo/python-list

Re: when can i expect libraries and third party tools to be updated for python 3 ?

[pruebauno]

On Apr 20, 9:47 am, Deep_Feelings  wrote:
> every one is telling "dont go with python 3 , 3rd party tools and
> libraries have no compitability with python 3"
>
> so from previous experience : when can i expect libraries and third
> party tools to be updated for python 3 ? (especially libraries )


I predict: one year.


(THIS PREDICTION IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
SHALL THE
AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
ARISING FROM,
OUT OF OR IN CONNECTION WITH THE PREDICTION OR THE USE OR OTHER
DEALINGS IN
THE PREDICTION.)

:-)
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python, MS SQL, and batch inserts

[pruebauno]

On Apr 21, 5:21 pm, Scott David Daniels  wrote:
> ericwoodwo...@gmail.com wrote:
> > On Apr 21, 4:01 pm, ericwoodwo...@gmail.com wrote:
> >> On Apr 21, 3:36 pm, Scott David Daniels  wrote:
> >>>  I forget the name of the SQL Server bulk loader, 
>
> bcp (bulk copy) was the name of the bulk loader.  I just remembered.
>
> Sorry, no more exciting advice.  it does sound like a good idea to go
> to a SQL Server group.
>
> --Scott David Daniels
> scott.dani...@acm.org

For MS SQL try:

BULK INSERT AdventureWorks.Sales.SalesOrderDetail
FROM '':\\''
WITH (ROWTERMINATOR = '''+CHAR(10)+''')
--
http://mail.python.org/mailman/listinfo/python-list

Help AIX 5.3 build on Python-3.1a2

[pruebauno]

OPT=-O2 LDFLAGS=-s ./configure --prefix=/ptst --with-gcc="xlc_r -q64"
--with-cxx="xlC_r -q64" --disable-ipv6 AR="ar -X64" --without-locale --
without-ctypes

checking for --with-universal-archs... 32-bit
checking MACHDEP... aix5
checking machine type as reported by uname -m... 00023AAA4C00
checking for --without-gcc... xlc_r -q64
checking for gcc... xlc_r -q64
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... no
checking whether xlc_r -q64 accepts -g... yes
checking for xlc_r -q64 option to accept ISO C89... none needed
checking for --with-cxx-main=... no
checking for c++... no
checking for g++... no
checking for gcc... no
checking for CC... no
checking for cxx... no
checking for cc++... no
checking for cl... no
checking how to run the C preprocessor... xlc_r -q64 -E
checking for grep that handles long lines and -e... /usr/bin/grep
checking for egrep... /usr/bin/grep -E
checking for AIX... yes
checking for --with-suffix...
checking for case-insensitive build directory... no
checking LIBRARY... libpython$(VERSION).a
checking LINKCC... $(srcdir)/Modules/makexp_aix Modules/python.exp . $
(LIBRARY); $(PURIFY) $(MAINCC)
checking for --enable-shared... no
checking for --enable-profiling...
checking LDLIBRARY... libpython$(VERSION).a
checking for ranlib... ranlib
checking for ar... ar -X64
checking for svnversion... not-found
checking for a BSD-compatible install... ./install-sh -c
checking for --with-pydebug... no
checking whether xlc_r -q64 accepts -OPT:Olimit=0... no
checking whether xlc_r -q64 accepts -Olimit 1500... no
checking whether pthreads are available without options... yes
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking asm/types.h usability... no
checking asm/types.h presence... no
checking for asm/types.h... no
checking conio.h usability... no
checking conio.h presence... no
checking for conio.h... no
checking curses.h usability... yes
checking curses.h presence... yes
checking for curses.h... yes
checking direct.h usability... no
checking direct.h presence... no
checking for direct.h... no
checking dlfcn.h usability... yes
checking dlfcn.h presence... yes
checking for dlfcn.h... yes
checking errno.h usability... yes
checking errno.h presence... yes
checking for errno.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking grp.h usability... yes
checking grp.h presence... yes
checking for grp.h... yes
checking ieeefp.h usability... no
checking ieeefp.h presence... no
checking for ieeefp.h... no
checking io.h usability... no
checking io.h presence... no
checking for io.h... no
checking langinfo.h usability... yes
checking langinfo.h presence... yes
checking for langinfo.h... yes
checking libintl.h usability... yes
checking libintl.h presence... yes
checking for libintl.h... yes
checking ncurses.h usability... no
checking ncurses.h presence... no
checking for ncurses.h... no
checking poll.h usability... yes
checking poll.h presence... yes
checking for poll.h... yes
checking process.h usability... no
checking process.h presence... no
checking for process.h... no
checking pthread.h usability... yes
checking pthread.h presence... yes
checking for pthread.h... yes
checking shadow.h usability... no
checking shadow.h presence... no
checking for shadow.h... no
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking for stdint.h... (cached) yes
checking stropts.h usability... yes
checking stropts.h presence... yes
checking for stropts.h... yes
checking termios.h usability... yes
checking termios.h presence... yes
checking for termios.h... yes
checking thread.h usability... no
checking thread.h presence... no
checking for thread.h... no
checking for unistd.h... (cached) yes
checking utime.h usability... yes
checking utime.h presence... yes
checking for utime.h... yes
checking sys/audioio.h usability... no
checking sys/audioio.h presence... no
checking for sys/audioio.h... no
checking sys/bsdtty.h usability... no
checking sys/bsdtty.h presence... no
checking for sys/bsdtty.h... no
checking sys/epoll.h usability... no
checking sys/epoll.h presence... no
checking for sys/epoll.h... no
checking sys/event.h usability... no
checking sys/event.h presence... no
checking for sys/event.h... no
checking sys/file.h usability... yes
checking sys/file.h presence... yes
checking for sys/file.h... yes
checking sys/loadavg.h usability... no
checking sys/loadavg.h presence... no
checking for sys/loadavg.h... no
checking sys/lock.h

Re: Help AIX 5.3 build on Python-3.1a2

[pruebauno]

On Apr 26, 5:14 am, Jeroen Ruigrok van der Werven  wrote:
> -On [20090425 19:17], Aahz (a...@pythoncraft.com) wrote:
>
> >In article 
> >,
> >  wrote:
> >>"Include/token.h", line 42.9: 1506-213 (S) Macro name TILDE cannot be
> >>redefined.
> >>"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
> >>250 of /usr/include/sys/ioctl.h.
>
> >Can you try trimming down the compilation to a small reproducible case?
>
> I thought it was already clear from what was provided?
>
> Include/token.h has a #define for TILDE and apparently AIX has a #define for
> TILDE in /usr/include/sys/ioctl.h as well. So you can a redefinition.
>
> One way around it, depending how AIX protects headers might be to change
> Include/token.h to either:
>
> #if defined(_SYS_IOCTL_H_)
> #undef TILDE
> #define TILDE           32
> #endif
>
> or
>
> #if defined(aix)
> #undef TILDE
> #define TILDE           32
> #endif
>
> --
> Jeroen Ruigrok van der Werven  / asmodai
> イェルーン ラウフロック ヴァン デル ウェルヴェンhttp://www.in-nomine.org/|http://www.rangaku.org/| 
> GPG: 2EAC625B
> A rose is a rose is a rose is a rose...

Thanks Jeroen. I know that AIX isn't as supported as other platforms,
but I thought it wouldn't hurt to ask anyway. At least now everybody
can search for that particular problem and find something. I will
experiment a little bit and see if I make it compile.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Help AIX 5.3 build on Python-3.1a2

[pruebauno]

On Apr 27, 10:26 am, Jeroen Ruigrok van der Werven  wrote:
> -On [20090427 15:00], prueba...@latinmail.com (prueba...@latinmail.com) wrote:
>
> >Thanks Jeroen. I know that AIX isn't as supported as other platforms,
> >but I thought it wouldn't hurt to ask anyway. At least now everybody
> >can search for that particular problem and find something. I will
> >experiment a little bit and see if I make it compile.
>
> I'll be interested to see if that fixes it. I have had to support a bunch of
> different Unix systems over the years compile-wise, but I don't think AIX
> was one of them though, so I might be a bit rusty there.
>
> --
> Jeroen Ruigrok van der Werven  / asmodai
> イェルーン ラウフロック ヴァン デル ウェルヴェンhttp://www.in-nomine.org/|http://www.rangaku.org/| 
> GPG: 2EAC625B
> What is the short meaning of this long speech..?

Yes, I tried them and either way fixes it. But this was with version 6
of the xlc compiler and it later gave me this error:

xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Python/symtable.o Python/symtable.c

"Python/symtable.c", line 767.50: 1506-068 (S) Operation between types
"struct _object*" and "int" is not allowed.
"Python/symtable.c", line 826.55: 1506-068 (S) Operation between types
"struct _object*" and "int" is not allowed.
make: 1254-004 The error code from the last command is 1.

I gave up and found a machine with xlc version 8 installed. Using that
version gave me a bunch of warnings instead but it compiled. Just for
the curious this is (more or less) what it looks like (long text
follows!):

(//ptst/Python-3.1a2) > make
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/acceler.o Parser/acceler.c
"Include/token.h", line 42.9: 1506-236 (W) Macro name TILDE has been
redefined.
"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
250 of /usr/include/sys/ioctl.h.
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/grammar1.o Parser/grammar1.c
"Include/token.h", line 42.9: 1506-236 (W) Macro name TILDE has been
redefined.
"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
250 of /usr/include/sys/ioctl.h.
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/listnode.o Parser/listnode.c
"Include/token.h", line 42.9: 1506-236 (W) Macro name TILDE has been
redefined.
"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
250 of /usr/include/sys/ioctl.h.
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/node.o Parser/node.c
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/parser.o Parser/parser.c
"Include/token.h", line 42.9: 1506-236 (W) Macro name TILDE has been
redefined.
"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
250 of /usr/include/sys/ioctl.h.
"Parser/parser.c", line 32.31: 1506-1298 (W) The subscript 1500 is out
of range. The valid range is 0 to 1499.
"Parser/parser.c", line 293.45: 1506-1298 (W) The subscript 1500 is
out of range. The valid range is 0 to 1499.
"Parser/parser.c", line 314.29: 1506-1298 (W) The subscript 1500 is
out of range. The valid range is 0 to 1499.
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/parsetok.o Parser/parsetok.c
"Include/token.h", line 42.9: 1506-236 (W) Macro name TILDE has been
redefined.
"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
250 of /usr/include/sys/ioctl.h.
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/bitset.o Parser/bitset.c
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/metagrammar.o Parser/metagrammar.c
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/firstsets.o Parser/firstsets.c
"Include/token.h", line 42.9: 1506-236 (W) Macro name TILDE has been
redefined.
"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
250 of /usr/include/sys/ioctl.h.
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/grammar.o Parser/grammar.c
"Include/token.h", line 42.9: 1506-236 (W) Macro name TILDE has been
redefined.
"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
250 of /usr/include/sys/ioctl.h.
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/pgen.o Parser/pgen.c
"Include/token.h", line 42.9: 1506-236 (W) Macro name TILDE has been
redefined.
"Include/token.h", line 42.9: 1506-358 (I) "TILDE" is defined on line
250 of /usr/include/sys/ioctl.h.
xlc_r -q64 -c  -DNDEBUG -O2  -I. -IInclude -I./Include   -
DPy_BUILD_CORE -o Parser/myreadline.o Parser/myreadline.c
xlc_r -q64 -c  -DNDEBUG -

Re: dict is really slow for big truck

[pruebauno]

On Apr 29, 1:05 pm, Scott David Daniels  wrote:
> Bruno Desthuilliers wrote:
> > d = {}
> > for line in open(thefile):
> >    arr = line.strip().split()
> >    d[arr[0]] = arr
>
> Sorry, not picking on Bruno in particular, but I keep seeing
> this formulation around various places.
> When does line.strip().split() ever differ from line.split()?
>
> --Scott David Daniels
> scott.dani...@acm.org

They don't.
It is probably out of habit of using the generalized idiom:
>>> line="a,b,c\n"
>>> line.strip().split(",")
['a', 'b', 'c']
>>> line.split(",")
['a', 'b', 'c\n']
--
http://mail.python.org/mailman/listinfo/python-list

Re: Help AIX 5.3 build on Python-3.1a2

[pruebauno]

On Apr 28, 10:07 am, Jeroen Ruigrok van der Werven  wrote:
> -On [20090427 20:31], prueba...@latinmail.com (prueba...@latinmail.com) wrote:
>
> >./Modules/ld_so_aix xlc_r -q64 -bI:Modules/python.exp build/
> >temp.aix-5.3-3.1//ptst/Python-3.1a2/Modules/_tkinter.o build/
> >temp.aix-5.3-3.1//ptst/Python-3.1a2/Modules/tkappinit.o -L/usr/X11R6/
> >lib64 -L/usr/X11R6/lib -L/usr/local/lib -ltk8.3 -ltcl8.3 -lX11 -o
> >build/lib.aix-5.3-3.1/_tkinter.so
> >ld: 0706-006 Cannot find or open library file: -l tk8.3
> >        ld:open(): A file or directory in the path name does not
> >exist.
>
> Well, do you have TK installed? It seems so, but are the paths passed to the
> compiler/linker correct (-L) for locating libtk8.3.so?
>
> --
> Jeroen Ruigrok van der Werven  / asmodai
> イェルーン ラウフロック ヴァン デル ウェルヴェンhttp://www.in-nomine.org/|http://www.rangaku.org/| 
> GPG: 2EAC625B
> Time is a twofold teacher, harsh and yet patient like no-one...

I don't think I have tk. Since it is the last thing to build it's not
a problem. At that point I just call it a successful compilation :-).
Python still works fine regardless.
--
http://mail.python.org/mailman/listinfo/python-list

Re: print(f) for files .. and is print % going away?

[pruebauno]

On Apr 30, 8:30 am, Esmail  wrote:
> Matt Nordhoff wrote:
> > Esmail wrote:
> >> Hello all,
>
> >> I use the print method with % for formatting my output to
> >> the console since I am quite familiar with printf from my
> >> C days, and I like it quite well.
>
> >> I am wondering if there is a way to use print to write
> >> formatted output to files?
>
> >> Also, it seems like I read that formatting with print is
> >> going away eventually and we should start using something
> >> else? Is that true and if so, what?
>
> >> Thanks,
> >> Esmail
>
> Hi Matt,
>
> > String formatting has nothing to do with the print statement/function.
> > It's an operator, just like doing "foo" + "bar"; you can use it wherever
> > you want.
>
> Ah .. so this means I could use this formatting with the
> write method for files .. that is great news (you don't
> want to see the ugly code I put together there .. :-)
>
> > See 
> > Also see  for
> > information on the replacement for the old string formatting, Python
> > 2.6's new str.format().
>
> Will do .. so do you think it's good to move to the new format,
> or will the older one still be around for a while?
>
> Thanks again!
>
> Esmail

There is also the Template class in the stdlib string module, for unix
shell/perl style "formatting".

The old mod (%) formatting will be around in 3.1 and that version not
even out yet, so it will be around for a couple more years at the very
least. Although the new format is more powerful and slightly less
error prone, there is so much old code using % that there is
resistance to drop it. The printf format is also well known among C
programmers. One thing many Python programmers don't like is that it
is a single character operator (%). It makes formatting hard to spot.
Maybe printf formatting will be moved to some module in the stdlib in
the future as a compromise.

I hope that the information will help you decide what to use.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.1 beta 1

[pruebauno]

On May 6, 9:32 pm, Benjamin Peterson  wrote:
> On behalf of the Python development team, I'm thrilled to announce the first 
> and
> only beta release of Python 3.1.
>
> Python 3.1 focuses on the stabilization and optimization of features and 
> changes
> Python 3.0 introduced.  For example, the new I/O system has been rewritten in 
> C
> for speed.  File system APIs that use unicode strings now handle paths with
> undecodable bytes in them. [1] Other features include an ordered dictionary
> implementation and support for ttk Tile in Tkinter.  For a more extensive list
> of changes in 3.1, seehttp://doc.python.org/dev/py3k/whatsnew/3.1.htmlor
> Misc/NEWS in the Python distribution.
>
> Please note that this is a beta release, and as such is not suitable for
> production environments.  We continue to strive for a high degree of quality,
> but there are still some known problems and the feature sets have not been
> finalized.  This beta is being released to solicit feedback and hopefully
> discover bugs, as well as allowing you to determine how changes in 3.1 might
> impact you.  If you find things broken or incorrect, please submit a bug 
> report
> at
>
>    http://bugs.python.org
>
> For more information and downloadable distributions, see the Python 3.1 
> website:
>
>    http://www.python.org/download/releases/3.1/
>
> See PEP 375 for release schedule details:
>
>    http://www.python.org/dev/peps/pep-0375/
>
> Enjoy,
> -- Benjamin
>
> Benjamin Peterson
> benjamin at python.org
> Release Manager
> (on behalf of the entire python-dev team and 3.1's contributors)


Congratulations!

Is it just me or was some nice summary output added to the make
process? I get a nice list of modules that didn't compile and the ones
where the library could not be found.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.1 beta 1

[pruebauno]

On May 7, 11:57 am, bearophileh...@lycos.com wrote:
> >Equality tests between OrderedDict objects are order-sensitive and are 
> >implemented as list(od1.items())==list(od2.items()). Equality tests between 
> >OrderedDict objects and other Mapping objects are order-insensitive<
>
> very nice idea.
>

I don't know if somebody else is interested but I wouldn't mind
support to OrderedDict and namedtuple in the csv module:

Have csv.DictReader return an OrderedDict instead of a regular one
(based on the order of the column names in the header line).

Have csv.DictWriter accept an OrderedDict. That would take care of
this:

"Note that unlike the DictReader class, the fieldnames parameter of
the DictWriter is not optional. Since Python’s dict objects are not
ordered, there is not enough information available to deduce the order
in which the row should be written to the csvfile."
http://docs.python.org/dev/py3k/library/csv.html#csv.DictWriter

Add a new csv.NamedTupleReader and csv.NamedTupleWriter that work
similar to the DictReader and DictWriter then this snippet (in
examples 
http://docs.python.org/dev/py3k/library/collections.html#collections.namedtuple
):

EmployeeRecord = namedtuple('EmployeeRecord', 'name, age, title,
department, paygrade')
import csv
for emp in map(EmployeeRecord._make, csv.reader(open("employees.csv",
"rb"))):
print(emp.name, emp.title)

Could be rewritten as:

import csv
for emp in csv.NamedTupleReader("employees.csv"):
print(emp.name, emp.title)


Also, is there a way to convert a OrderedDict into a namedtuple? It is
possible to go the other direction using namedtuple._asdict, but I
don't see a _fromdict method.

And thanks Steven and Raymond for the Counter class. Several people
had started to see the common pattern and now we have it in the
stdlib.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 3.1 beta 1

[pruebauno]

On May 7, 6:33 pm, Benjamin Peterson  wrote:
>   latinmail.com> writes:
>
> > Congratulations!
>
> Thanks!
>
>
>
> > Is it just me or was some nice summary output added to the make
> > process? I get a nice list of modules that didn't compile and the ones
> > where the library could not be found.
>
> Are you compiling on a different platform? The nice output has been around 
> for a
> while, bu only on non-Windows platforms.

Not really, I was on AIX. It is probably just me then, probably
because for the first time I see it making it through the whole
process even with the Tkinter libraries missing.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Python 2.6, File read() problems in Windows Xp

[pruebauno]

On May 8, 5:08 am, Li Wang  wrote:
> Hi  Dave:
> Thank you very much for you explanation:)
>
> > Chances are you forgot the "b" parameter to open().  Unnecessary in Unix, it
> > tells the library to *not* translate \r\n  to \n upon read, or the inverse
> > on write.  In other words, with the "b" parameter, the file is read in
> > unchanged.
>
> So, if I am using python in Linux, do open('file', 'r') and
> open('file', 'rb') work the same way?
>
> Thanks,
>
> Best regards,
> Li

In old Python up to 2.6, YES.
--
http://mail.python.org/mailman/listinfo/python-list

Re: I'm intrigued that Python has some functional constructions in the language.

[pruebauno]

On May 8, 3:04 pm, Casey Hawthorne  wrote:
> I'm intrigued that Python has some functional constructions in the
> language.
>
> Would it be possible to more clearly separate the pure code (without
> side effects) from the impure code (that deals with state changes,
> I/O, etc.), so that the pure code could be compiled and have
> aggressive functional transformations applied to it for efficiency.
>
> That way, the syntax would be a lot easier to understand, than most
> functional languages, like Haskell.
>
> I gave a presentation at the beginning of last year on Haskell and at
> the end, someone's comment was, "I can see the benefits of functional
> programming but why does it have to be so cryptic."
> --
> Regards,
> Casey

Don't forget that the Python interpreter is simple. It makes
maintenance easier and allows embedding it into other programs. Good
optimizing compilers for functional languages are not simple. Your
idea would be something that could be added to the PyPy project in the
future.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Is there a better way to chose a slice of a list?

[pruebauno]

On May 8, 3:03 pm, walterbyrd  wrote:
> This works, but it seems like there should be a better way.
>
> --
> week = ['sun','mon','tue','wed','thu','fri','sat']
> for day in week[week.index('tue'):week.index('fri')]:
>    print day
> ---

Depending on the context this style might help:

>>> week = ['sun','mon','tue','wed','thu','fri','sat']
>>> tue_fri = slice(week.index('tue'), week.index('fri'))
>>> for day in week[tue_fri]:
print day

But I don't really see it as an improvement unless you are using those
intervals repeatedly.
--
http://mail.python.org/mailman/listinfo/python-list

Illegal seek with os.popen

[pruebauno]

Should I open a bug report for this?

Python 2.5.1 (r251:54863, Sep 19 2007, 14:58:06) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.popen('cat','w')

>>>

Python 3.1rc1 (r31rc1:73054, Jun  1 2009, 10:49:24) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> os.popen('cat','w')
Traceback (most recent call last):
  File "", line 1, in 
  File "/Python-3.1rc1/Lib/os.py", line 641, in popen
return _wrap_close(io.TextIOWrapper(proc.stdin), proc)
IOError: [Errno 29] Illegal seek
>>>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Illegal seek with os.popen

[pruebauno]

On Jun 3, 3:36 pm, a...@pythoncraft.com (Aahz) wrote:
> In article 
> <7c93031a-235e-4e13-bd37-7c9dbc6e8...@r16g2000vbn.googlegroups.com>,
>
>
>
>   wrote:
> >Should I open a bug report for this?
>
> >Python 2.5.1 (r251:54863, Sep 19 2007, 14:58:06) [C] on aix5
> >Type "help", "copyright", "credits" or "license" for more information.
>  import os
>  os.popen('cat','w')
> >
>
> >Python 3.1rc1 (r31rc1:73054, Jun  1 2009, 10:49:24) [C] on aix5
> >Type "help", "copyright", "credits" or "license" for more information.
>  os.popen('cat','w')
> >Traceback (most recent call last):
> >  File "", line 1, in 
> >  File "/Python-3.1rc1/Lib/os.py", line 641, in popen
> >    return _wrap_close(io.TextIOWrapper(proc.stdin), proc)
> >IOError: [Errno 29] Illegal seek
>
> What happens in 2.6 and 3.0?
> --
> Aahz (a...@pythoncraft.com)           <*>        http://www.pythoncraft.com/
>
> "Given that C++ has pointers and typecasts, it's really hard to have a
> serious conversation about type safety with a C++ programmer and keep a
> straight face.  It's kind of like having a guy who juggles chainsaws
> wearing body armor arguing with a guy who juggles rubber chickens wearing
> a T-shirt about who's in more danger."  --Roy Smith, c.l.py, 2004.05.23

Python 2.6.2 (r262:71600, Jun  4 2009, 16:07:26) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.popen('cat','w')

>>>

Python 3.0.1 (r301:69556, Jun  4 2009, 16:07:22) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.popen('cat','w')


So it seems to be something in 3.1 that causes it to fail.
BTW it is not like I use os.popen a lot. I found this problem while
trying to use help().

Python 3.1rc1 (r31rc1:73054, Jun  1 2009, 10:49:24) [C] on aix5
Type "help", "copyright", "credits" or "license" for more information.
>>> help(open)
Traceback (most recent call last):
  File "", line 1, in 
  File "/Python-3.1rc1/Lib/site.py", line 429, in __call__
return pydoc.help(*args, **kwds)
  File "/Python-3.1rc1/Lib/pydoc.py", line 1709, in __call__
self.help(request)
  File "/Python-3.1rc1/Lib/pydoc.py", line 1756, in help
else: doc(request, 'Help on %s:')
  File "/Python-3.1rc1/Lib/pydoc.py", line 1505, in doc
pager(render_doc(thing, title, forceload))
  File "/Python-3.1rc1/Lib/pydoc.py", line 1320, in pager
pager(text)
  File "/Python-3.1rc1/Lib/pydoc.py", line 1340, in 
return lambda text: pipepager(text, 'less')
  File "/Python-3.1rc1/Lib/pydoc.py", line 1359, in pipepager
pipe = os.popen(cmd, 'w')
  File "/Python-3.1rc1/Lib/os.py", line 641, in popen
return _wrap_close(io.TextIOWrapper(proc.stdin), proc)
IOError: [Errno 29] Illegal seek
>>>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Native Code vs. Python code for modules

[pruebauno]

On Jul 29, 11:56 pm, koblas <[EMAIL PROTECTED]> wrote:
> better if there was a way that if I have an "interface compatible"
> native (aka C) module that has better performance that there could be
> a way that python would give it preference.
>
> e.g.
>
>   import random(version=1.2, lang=c)
> or
>   import random(version=1.2, lang=py)   # use the python version by
> default
> or
>   import random #  use the latest version in the "fastest" code (C
> given preference)
>
> where there could be a nice set of "standard" key value pairs that
> could provide addtional hints as to what language and version of a
> library was to be used.

I will only make a comment on automatic cModule importing. Some of
what you are suggesting already happens (e.g. Elementree). In Python
3.0 more of the "legacy" modules (e.g. StringIO) should also work that
way. I don't know what the current status for Pickle is  because
cPickle had some additional internal functionality not available in
regular Pickle that Zope was dependent on.
--
http://mail.python.org/mailman/listinfo/python-list

Re: List of modules available for import inside Python?

[pruebauno]

On Aug 28, 12:21 am, ssecorp <[EMAIL PROTECTED]> wrote:
> Is there a way to view all the modules I have available for import
> from within Python?
> Like writing in the interpreter:
> import.modules
>
> Also, is there anything like Cpan for Python?

Isn't the most obvious answer to the first question this link?

http://docs.python.org/modindex.html
--
http://mail.python.org/mailman/listinfo/python-list

Re: improving a huge double-for cycle

[pruebauno]

On Sep 18, 8:25 am, Alexzive <[EMAIL PROTECTED]> wrote:
> Hello there :) ,
>
> I am a python newbie and need to run following code for a task in an
> external simulation programm called "Abaqus" which makes use of python
> to access the mesh (ensamble of nodes with xy coordinates) of a
> certain geometrical model.
>
> [IN is the starting input containing the nodes to be check, there are
> some double nodes with the same x and y coordinates which need to be
> removed. SN is the output containing such double nodes]
>
> Code: Select all
> for i in range(len(IN)): #scan all elements of the list IN
>   for j in range(len(IN)):
> if i <> j:
>  if IN[i].coordinates[0] == IN[j].coordinates[0]:
>if IN[i].coordinates[1] == IN[j].coordinates[1]:
>   SN.append(IN[i].label)
>
> Unfortunately my len(IN) is about 100.000 and the running time about
> 15h  :(
>
> Any idea to improve it?
>
> I have already tried to group the "if statements" in a single one:
>
> Code: Select all
> if i <> j and if IN[i].coordinates[0] == IN[j].coordinates[0] and
> if IN[i].coordinates[1] == IN[j].coordinates[1]:
>
> but no improvements.
>
> Many thanks, Alex


dup=set()
SN=[]
for item in IN:
   c=item.coordinates[0], item.coordinates[1]
   if c in dup:
  SN.append(item.label)
   else:
  dup.add(c)
--
http://mail.python.org/mailman/listinfo/python-list

Re: dict generator question

[pruebauno]

On Sep 18, 10:54 am, "Simon Mullis" <[EMAIL PROTECTED]> wrote:
> Hi,
>
> Let's say I have an arbitrary list of minor software versions of an
> imaginary software product:
>
> l = [ "1.1.1.1", "1.2.2.2", "1.2.2.3", "1.3.1.2", "1.3.4.5"]
>
> I'd like to create a dict with major_version : count.
>
> (So, in this case:
>
> dict_of_counts = { "1.1" : "1",
>"1.2" : "2",
>"1.3" : "2" }
>
> Something like:
>
> dict_of_counts = dict([(v[0:3], "count") for v in l])
>
> I can't seem to figure out how to get "count", as I cannot do x += 1
> or x++ as x may or may not yet exist, and I haven't found a way to
> create default values.
>
> I'm most probably not thinking pythonically enough... (I know I could
> do this pretty easily with a couple more lines, but I'd like to
> understand if there's a way to use a dict generator for this).
>
> Thanks in advance
>
> SM
>
> --
> Simon Mullis

3 lines:

from collections import defaultdict
dd=defaultdict(int)
for x in l: dd[x[0:3]]+=1
--
http://mail.python.org/mailman/listinfo/python-list

Re: dict generator question

[pruebauno]

On Sep 18, 10:54 am, "Simon Mullis" <[EMAIL PROTECTED]> wrote:
> Hi,
>
> Let's say I have an arbitrary list of minor software versions of an
> imaginary software product:
>
> l = [ "1.1.1.1", "1.2.2.2", "1.2.2.3", "1.3.1.2", "1.3.4.5"]
>
> I'd like to create a dict with major_version : count.
>
> (So, in this case:
>
> dict_of_counts = { "1.1" : "1",
>"1.2" : "2",
>"1.3" : "2" }
>
> Something like:
>
> dict_of_counts = dict([(v[0:3], "count") for v in l])
>
> I can't seem to figure out how to get "count", as I cannot do x += 1
> or x++ as x may or may not yet exist, and I haven't found a way to
> create default values.
>
> I'm most probably not thinking pythonically enough... (I know I could
> do this pretty easily with a couple more lines, but I'd like to
> understand if there's a way to use a dict generator for this).
>
> Thanks in advance
>
> SM
>
> --
> Simon Mullis

Considering 3 identical "simultpost" solutions I'd say:
 "one obvious way to do it" FTW :-)
--
http://mail.python.org/mailman/listinfo/python-list

Re: improving a huge double-for cycle

[pruebauno]

On Sep 18, 7:42 pm, Steven D'Aprano <[EMAIL PROTECTED]
cybersource.com.au> wrote:
> On Thu, 18 Sep 2008 20:43:00 +0200, Bruno Desthuilliers wrote:
> > Now the obvious winner is pruebono - even unoptimized, using sets seems
> > to be *way* faster than even the most optimized corrected version of
> > your algorithm.
>
> I'm not being snarky about losing priority here, but I submitted
> essentially the same solution two hours earlier than pruebono.
>
> Are people not seeing my posts? Have I been kill-filed by everyone except
> Mensator? I also asked a question about HTTPError and I haven't seen any
> responses at all.
>
> --
> Steven

I see your post now, but I didn't notice it at the time I posted.
Could be because I am using the noticeable bad Google groups interface
that is known for being unreliable. Duplicate solutions on Usenet are
almost a given and I consider duplicate solutions a good thing, it
means that other people will be able to understand that code.

In any case I am not here for glory I am posting under a pseudonym so
nobody discovers that I am slacking off at work reading Usen[carrier
lost]
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to debug python code?

[pruebauno]

[EMAIL PROTECTED] wrote:
> hi,
>I am new to Python programming.I am not getting exactly pdb.Can
> anyone tell me effective  way to debug python code?
>Please give me any example.
>Looking for responce.
>Thank You.
> Sushant

If you are having issues you also might want to try:
http://pychecker.sourceforge.net/
and see if it finds your problem. You might get lucky.

Or just make the bigger install:
http://stani.be/python/spe/blog/
that one already includes pychecker and a python debugger.

Sorry for the noise if you were just looking for instructions for pdb.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: "definitive" source on advanced python?

[pruebauno]

vdrab wrote:
> Hi all,
>
> Is there some sort of coherent source (dead tree format, maybe?) on
> some of the more advanced features
> of python (decorators, metaclasses, etc)? I'm sort of looking for a

If you just want a good  book in feature description I recomend Python
in a Nutshell. It will explain everything you ever did (or didn't) want
to know without much fuzz. It will not have a long list of when you
should use them. For the later the Python Cookbook is probably more
appropiate. It is the book with the most amount and kind of advanced
stuff IMHO. The printed version is much better than the online because
it not only contains the examples but explains how they work. The
problem is that the cookbook is organized by problem domain and not by
implementation technique. To learn techniques you have to read it like
the Bible: you read the stories and try to deduce what it tells you
about Python.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: fastest way to test file for string?

[pruebauno]

On Jun 5, 7:50 am, kj  wrote:
> Hi.  I need to implement, within a Python script, the same
> functionality as that of Unix's
>
>    grep -rl some_string some_directory
>
> I.e. find all the files under some_directory that contain the string
> "some_string".
>
> I imagine that I can always resort to the shell for this, but is
> there an efficient way of doing it within Python?
>
> (BTW, portability is not high on my list here; this will run on a
> Unix system, and non-portable Unix-specific solutions are fine with
> me.)
>
> TIA!
> --

You can write your own version of grep in python using os.walk, open,
read and find. I don't know why one would want to do that unless for
portability reasons. It will be pretty hard to beat grep in efficiency
and succinctness. The most sensible thing IMHO is a shell script or
call grep using os.system (or using subprocess).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: DUDA !!!!!!!!!!

[pruebauno]

On Jun 4, 2:51 pm, "Ariel Vazquez Riveron" 
wrote:
> Hola:
>   Hoy en día me encuentro iniciandome dentro del python, en estos  
> momentos quiero saber de que forma puedo eliminar un archivo de un  
> compactado, ya sea zip, rar o cualquier otro. Estudie las librerías  
> zipfile pero no tiene ninguna funcion que me permita hacerlo. Trabajo  
> con python 2.5
>
> salu2 Ariel
>
> 
> This message was sent using IMP, the Internet Messaging Program.

En esta lista la mayoria no entiende castellano sino ingles. Mejor
pregunta en esta lista:

http://dir.gmane.org/gmane.org.user-groups.python.argentina
-- 
http://mail.python.org/mailman/listinfo/python-list