Re: Warning in python file when i m using pychecker.

[Fábio Santos]

Read the warnings carefully. They are pretty clear.
On 26 Apr 2013 07:48, "Avnesh Shakya"  wrote:

> hi,
>I am trying to run my file using pychecker, but it's showing warning. I
> am unable to get these warning. Please help me, how to remove these
> warning. I am using pychecker first time.
>
> avin@HP:~/github/UdacitySiteData$ pychecker udacity_to_jsonFinal.py
> Processing module udacity_to_jsonFinal (udacity_to_jsonFinal.py)...
>
> Warnings...
>
> [system path]/dist-packages/bs4/__init__.py:206: Parameter (successor) not
> used
> [system path]/dist-packages/bs4/__init__.py:209: Parameter (successor) not
> used
> [system path]/dist-packages/bs4/__init__.py:213: Local variable (tag) not
> used
>
> [system path]/dist-packages/bs4/element.py:306: Parameter (kwargs) not used
> [system path]/dist-packages/bs4/element.py:507: (id) shadows builtin
> [system path]/dist-packages/bs4/element.py:791: (next) shadows builtin
> [system path]/dist-packages/bs4/element.py:903: Invalid arguments to
> (__repr__), got 2, expected 1
>
> Thanks.
> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Warning in python file when i m using pychecker.

[Dave Angel]

On 04/26/2013 02:42 AM, Avnesh Shakya wrote:

hi,
I am trying to run my file using pychecker, but it's showing warning. I am 
unable to get these warning. Please help me, how to remove these warning. I am 
using pychecker first time.

avin@HP:~/github/UdacitySiteData$ pychecker udacity_to_jsonFinal.py
Processing module udacity_to_jsonFinal (udacity_to_jsonFinal.py)...

Warnings...

[system path]/dist-packages/bs4/__init__.py:206: Parameter (successor) not used
[system path]/dist-packages/bs4/__init__.py:209: Parameter (successor) not used
[system path]/dist-packages/bs4/__init__.py:213: Local variable (tag) not used

[system path]/dist-packages/bs4/element.py:306: Parameter (kwargs) not used
[system path]/dist-packages/bs4/element.py:507: (id) shadows builtin
[system path]/dist-packages/bs4/element.py:791: (next) shadows builtin
[system path]/dist-packages/bs4/element.py:903: Invalid arguments to 
(__repr__), got 2, expected 1

Thanks.



What's not clear about any of these?  On line 206, you defined some 
function and one of the formal parameters was called successor.  You 
don't use that value anywhere in the function.  The fix?  Omit the 
unused parameter in the function definition, and change all the caller 
sites.  Sometimes this is impractical, for example in functions that are 
callbacks from code you don't control.


Likewise for the next 3.

id() is a built-in function, so you shouldn't use that name for your own 
functions or variables.  rename it.


Likewise next, which is even more likely to cause confusion, since next 
is frequently used in real code, while id() is more likely a debugging aid.


If you want anything more detailed, perhaps you should show the context 
around each warning.  You have the line numbers, so that should be easy.



--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: Weird python behavior

[rusi]

On Apr 26, 11:25 am, rusi  wrote:
> To present these kind of errors, Erlang has a concept of sticky
> modules -- those that come from the system…

??present?? should have been 'prevent'
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: baffled classes within a function namespace. Evaluation order.

[Peter Otten]

Steven D'Aprano wrote:

> On Fri, 26 Apr 2013 07:39:32 +0200, Peter Otten wrote:
> 
>> A class body is basically a function body that is executed once. To
>> allow an assignment
>> 
> x = 42
> class A:
>> ... x = x
>> ...
>> 
>> which is not possible inside a function
> 
> 
> As far as I can tell, the documentation says that this assignment should
> also be impossible inside a class. Unless I'm missing something, I think
> this is a bug.
> 
> http://docs.python.org/3/reference/executionmodel.html
 
Will have a look.
 
> [...]
>> However, while the above gives some technical background it doesn't
>> explain why this scheme was chosen. My guess is that when Python got
>> closures nobody was willing to do the extra work to make class bodies
>> namespace-aware. The alternative, disallowing x = x in classes, would
>> have seriously broken backwards-compatibility.
> 
> Not so. x = x was not allowed in classes before there were closures:
> 
> 
> [steve@ando ~]$ python1.5
> Python 1.5.2 (#1, Aug 27 2012, 09:09:18)  [GCC 4.1.2 20080704 (Red Hat
> 4.1.2-52)] on linux2
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam


 def f():
> ... x = 1
> ... class Test:
> ... x = x
> ... return Test
> ...
 T = f()
> Traceback (innermost last):
>   File "", line 1, in ?
>   File "", line 3, in f
>   File "", line 4, in Test
> NameError: x

Define a global variable x and run it again. I don't have an ancient Python 
lying around but my "prediction" is

>>> x = 42
>>> f().x
42

which would be the same behaviour as that of Python 3.3.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Fixing escaped characters python-xbee

[pabloblo85]

> 
> the following in gmane.comp.python.general:
> 
> 
> 
> > I am using a XBee to receive data from an arduino network.
> 
> > 
> 
> > But they have AP=2 which means escaped characters are used when a 11 or 13 
> > appears (and some more...)
> 
> > 
> 
> > When this occurs, XBee sends 7D and inmediatly XOR operation with char and 
> > 0x20.
> 
> > 
> 
> > I am trying to recover the original character in python but I don't know ho 
> > to do it.
> 
> > 
> 
> > I tried something like this:
> 
> > 
> 
> > read = ser.read(4) #Read 4 chars from serial port
> 
> 
> 
>   Why read 4 at a time if you need to detect the escape marker...
> 
> 
> 
> PSEUDO_CODE -- UNTESTED:
> 
> 
> 
>   for c in ser.read():#presumes it will function as an iterator
> 
>   if ord(c) == 0x7D:
> 
>   c =chr(ord(ser.read(1)) ^ 0x20)
> 
>   #do something with c (save to a list for later joining as a
> 
> string?)
> 
>   #probably need some condition to exit the read loop too
> 
> > def logical_xor(str1, str2):
> 
> > return bool(str1) ^ bool(str2)
> 
> >
> 
>   bool() returns True or False based on the argument... Any non-empty
> 
> string will be True. Instead what you want is to x-or the bits of the
> 
> character itself.
> 
> -- 

It works! Thank you so much. Now I can go ahead with my work!
-- 
http://mail.python.org/mailman/listinfo/python-list

Nested For loop not running full

[inshu chauhan]

Hello everyone,

I have this part of my code where I am trying to traverse over an image by
running a for loop for both x and y co-ordinate axis. But the loop is
terminating by just reading first pixel. Can think of a reason why this is
happening ?

The code is:
for sy in xrange(0, segimage.height):
for sx in xrange(0, segimage.width):
if segimage[sy,sx] == (0.0, 0.0, 0.0):
continue
else:
seg_color = segimage[sy,sx]
blue = int(seg_color[0])
green = int(seg_color[1])
red = int(seg_color[2])
reg_num = blue + 256 * green + 65536 * red
for l in f:
sp = l.split(",")
if len(sp) == 14:
print sy, sx  # for checking which pixel its
reading currently
print reg_num, sp[0]  # for checking whats
happening
if reg_num == int(sp[0].strip()):
print reg_num, sp[0].strip() # for checking
whats happening
classification = int(sp[13].strip())


The inside "for loop" is for reading a csv format file from which I am
extracting some information.

Thanks in Advance for your suggestions
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Comparison Style

[Mark Lawrence]

On 25/04/2013 21:35, Steve Simmons wrote:



The Ying Tong song - a classic of its time. But eminently suited to the
chorally challenged.



Released on a classic EP with Major Dennis Bloodnok's Rock and Roll Call 
Rumba, I'm walking Backwards for Christmas and Bluebottle Blues.


Bravado, bravado. What a voice! (What a bank balance!)

--
If you're using GoogleCrap™ please read this 
http://wiki.python.org/moin/GoogleGroupsPython.


Mark Lawrence

--
http://mail.python.org/mailman/listinfo/python-list

Re: Nested For loop not running full

[Peter Otten]

inshu chauhan wrote:

> I have this part of my code where I am trying to traverse over an image by
> running a for loop for both x and y co-ordinate axis. But the loop is
> terminating by just reading first pixel. Can think of a reason why this is
> happening ?
> 
> The code is:
> for sy in xrange(0, segimage.height):
> for sx in xrange(0, segimage.width):
> if segimage[sy,sx] == (0.0, 0.0, 0.0):
> continue
> else:
> seg_color = segimage[sy,sx]
> blue = int(seg_color[0])
> green = int(seg_color[1])
> red = int(seg_color[2])
> reg_num = blue + 256 * green + 65536 * red
> for l in f:
> sp = l.split(",")
> if len(sp) == 14:
> print sy, sx  # for checking which pixel its
> reading currently
> print reg_num, sp[0]  # for checking whats
> happening
> if reg_num == int(sp[0].strip()):
> print reg_num, sp[0].strip() # for checking
> whats happening
> classification = int(sp[13].strip())
> 
> 
> The inside "for loop" is for reading a csv format file from which I am
> extracting some information.

My crystal ball says that the 'for sy...' and 'for sx...' loops are running 
to completion, but you don't get the coordinates printed because you put 
them into the 'for l in f' loop which will only run once.

The quick and dirty fix is to replace

f = open(...)

in the code you are not showing with 

f == list(open(...))

The reasonable thing to do is of course to move the preprocessing (e.g. csv-
parsing) out of the sy and sx loops.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested For loop not running full

[Terry Jan Reedy]

On 4/26/2013 4:48 AM, inshu chauhan wrote:

Hello everyone,

I have this part of my code where I am trying to traverse over an image
by running a for loop for both x and y co-ordinate axis. But the loop is
terminating by just reading first pixel. Can think of a reason why this
is happening ?


*A* reason could be that segimage.height and .width are both 1. I would 
print them out to see what they are.




The code is:
for sy in xrange(0, segimage.height):
 for sx in xrange(0, segimage.width):
 if segimage[sy,sx] == (0.0, 0.0, 0.0):
 continue
 else:
 seg_color = segimage[sy,sx]
 blue = int(seg_color[0])
 green = int(seg_color[1])
 red = int(seg_color[2])
 reg_num = blue + 256 * green + 65536 * red
 for l in f:
 sp = l.split(",")
 if len(sp) == 14:
 print sy, sx  # for checking which pixel its
reading currently
 print reg_num, sp[0]  # for checking whats
happening
 if reg_num == int(sp[0].strip()):
 print reg_num, sp[0].strip() # for checking
whats happening
 classification = int(sp[13].strip())

The inside "for loop" is for reading a csv format file from which I am
extracting some information.

Thanks in Advance for your suggestions







--
http://mail.python.org/mailman/listinfo/python-list

Re: Nested For loop not running full

[inshu chauhan]

On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pete...@web.de> wrote:

> inshu chauhan wrote:
>
> > I have this part of my code where I am trying to traverse over an image
> by
> > running a for loop for both x and y co-ordinate axis. But the loop is
> > terminating by just reading first pixel. Can think of a reason why this
> is
> > happening ?
> >
> > The code is:
> > for sy in xrange(0, segimage.height):
> > for sx in xrange(0, segimage.width):
> > if segimage[sy,sx] == (0.0, 0.0, 0.0):
> > continue
> > else:
> > seg_color = segimage[sy,sx]
> > blue = int(seg_color[0])
> > green = int(seg_color[1])
> > red = int(seg_color[2])
> > reg_num = blue + 256 * green + 65536 * red
> > for l in f:
> > sp = l.split(",")
> > if len(sp) == 14:
> > print sy, sx  # for checking which pixel its
> > reading currently
> > print reg_num, sp[0]  # for checking whats
> > happening
> > if reg_num == int(sp[0].strip()):
> > print reg_num, sp[0].strip() # for checking
> > whats happening
> > classification = int(sp[13].strip())
> >
> >
> > The inside "for loop" is for reading a csv format file from which I am
> > extracting some information.
>
> My crystal ball says that the 'for sy...' and 'for sx...' loops are running
> to completion, but you don't get the coordinates printed because you put
> them into the 'for l in f' loop which will only run once.
>

Is there any means by which I can run this 'For l in f' loop again and
again ?

>
> The quick and dirty fix is to replace
>
> f = open(...)
>
> in the code you are not showing with
>
> f == list(open(...))
>

f is just a text file(csv format).. so why list ??


> The reasonable thing to do is of course to move the preprocessing (e.g.
> csv-
> parsing) out of the sy and sx loops.
>

I did this but again then what I intend to do is not really happening, For
every pixel I read,  I want to traverse the full file, so that the
information I am taking from pixel have to match in one of the line in the
file. Can this be done by modifying my code ? or something new has to be
devised ?




> --
> http://mail.python.org/mailman/listinfo/python-list
>
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested For loop not running full

[Oscar Benjamin]

On 26 April 2013 10:36, inshu chauhan  wrote:
>
> On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pete...@web.de> wrote:
>>
>> My crystal ball says that the 'for sy...' and 'for sx...' loops are
>> running
>> to completion, but you don't get the coordinates printed because you put
>> them into the 'for l in f' loop which will only run once.
>
> Is there any means by which I can run this 'For l in f' loop again and again
> ?
>>
>> The quick and dirty fix is to replace
>>
>> f = open(...)
>>
>> in the code you are not showing with
>>
>> f == list(open(...))
>
> f is just a text file(csv format).. so why list ??

So that you can run the for l in f loop again and again. You can loop
over a list as many times as you like but only once over a file
(unless you reset the file pointer).


Oscar
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested For loop not running full

[Chris Angelico]

On Fri, Apr 26, 2013 at 7:36 PM, inshu chauhan  wrote:
>
> On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pete...@web.de> wrote:
>>
>> f = open(...)
>>
>> in the code you are not showing with
>>
>> f == list(open(...))
>
> f is just a text file(csv format).. so why list ??

(That should be =, not ==)

Instead of having an open file object, you would instead have a list
of the lines in the file. That can be iterated over more than once.

>> The reasonable thing to do is of course to move the preprocessing (e.g.
>> csv-
>> parsing) out of the sy and sx loops.
>
>
> I did this but again then what I intend to do is not really happening, For
> every pixel I read,  I want to traverse the full file, so that the
> information I am taking from pixel have to match in one of the line in the
> file. Can this be done by modifying my code ? or something new has to be
> devised ?

How large is the file? There are two easy solutions:

1) Open and close the file every time you touch a pixel
2) Open the file once, read it all into memory, and then iterate over
the in-memory copy every pixel

If your file is insanely large then the first option may be better,
but for anything less than five yottabytes, go with the second. (Okay,
I may be exaggerating slightly... let's say anything less than half
your RAM. So if you have 10YB of memory, then I wasn't exaggerating.)
That's why Peter suggested creating a list; you iterate over the list.
Another way to do it is to parse the file once and retain a more
efficient and useful structured form of the data... which is the other
thing Peter suggested ("move the preprocessing (e.g. csv-
parsing) out of the sy and sx loops").

So, yeah. Listen to Peter Otten, he knows what he's talking about :)

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested For loop not running full

[Peter Otten]

inshu chauhan wrote:

> On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pete...@web.de> wrote:
> 
>> inshu chauhan wrote:
>>
>> > I have this part of my code where I am trying to traverse over an image
>> by
>> > running a for loop for both x and y co-ordinate axis. But the loop is
>> > terminating by just reading first pixel. Can think of a reason why this
>> is
>> > happening ?
>> >
>> > The code is:
>> > for sy in xrange(0, segimage.height):
>> > for sx in xrange(0, segimage.width):
>> > if segimage[sy,sx] == (0.0, 0.0, 0.0):
>> > continue
>> > else:
>> > seg_color = segimage[sy,sx]
>> > blue = int(seg_color[0])
>> > green = int(seg_color[1])
>> > red = int(seg_color[2])
>> > reg_num = blue + 256 * green + 65536 * red
>> > for l in f:
>> > sp = l.split(",")
>> > if len(sp) == 14:
>> > print sy, sx  # for checking which pixel its
>> > reading currently
>> > print reg_num, sp[0]  # for checking whats
>> > happening
>> > if reg_num == int(sp[0].strip()):
>> > print reg_num, sp[0].strip() # for checking
>> > whats happening
>> > classification = int(sp[13].strip())
>> >
>> >
>> > The inside "for loop" is for reading a csv format file from which I am
>> > extracting some information.
>>
>> My crystal ball says that the 'for sy...' and 'for sx...' loops are
>> running to completion, but you don't get the coordinates printed because
>> you put them into the 'for l in f' loop which will only run once.
>>
> 
> Is there any means by which I can run this 'For l in f' loop again and
> again ?
> 
>>
>> The quick and dirty fix is to replace
>>
>> f = open(...)
>>
>> in the code you are not showing with
>>
>> f == list(open(...))
>>
> 
> f is just a text file(csv format).. so why list ??

Can you figure it out yourself from the following?

>>> f = open("tmp.data")
>>> for i in range(3):
... for line in f: print repr(line)
... 
'alpha\n'
'beta\n'
'gamma\n'
>>> f = list(open("tmp.data"))
>>> for i in range(3):
... for line in f: print repr(line)
... 
'alpha\n'
'beta\n'
'gamma\n'
'alpha\n'
'beta\n'
'gamma\n'
'alpha\n'
'beta\n'
'gamma\n'

>> The reasonable thing to do is of course to move the preprocessing (e.g.
>> csv-
>> parsing) out of the sy and sx loops.
>>
> 
> I did this but again then what I intend to do is not really happening, For
> every pixel I read,  I want to traverse the full file, so that the
> information I am taking from pixel have to match in one of the line in the
> file. Can this be done by modifying my code ? or something new has to be
> devised ?

I think I have already answered this, but here's another alternative:

>>> f = open("tmp.data")
>>> for i in range(3):
... f.seek(0)
... for line in f: print(repr(line))
... 
'alpha\n'
'beta\n'
'gamma\n'
'alpha\n'
'beta\n'
'gamma\n'
'alpha\n'
'beta\n'
'gamma\n'


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested For loop not running full

[Jens Thoms Toerring]

inshu chauhan  wrote:
> I have this part of my code where I am trying to traverse over an image by
> running a for loop for both x and y co-ordinate axis. But the loop is
> terminating by just reading first pixel. Can think of a reason why this is
> happening ?

> The code is:
> for sy in xrange(0, segimage.height):
> for sx in xrange(0, segimage.width):
> if segimage[sy,sx] == (0.0, 0.0, 0.0):
> continue
> else:
> seg_color = segimage[sy,sx]
> blue = int(seg_color[0])
> green = int(seg_color[1])
> red = int(seg_color[2])
> reg_num = blue + 256 * green + 65536 * red
> for l in f:
> sp = l.split(",")
> if len(sp) == 14:
> print sy, sx  # for checking which pixel its
> reading currently
> print reg_num, sp[0]  # for checking whats
> happening
> if reg_num == int(sp[0].strip()):
> print reg_num, sp[0].strip() # for checking
> whats happening
> classification = int(sp[13].strip())


> The inside "for loop" is for reading a csv format file from which I am
> extracting some information.

Are you sure that the loop is only run once? In that case the most
likely thing is that the image consists of only a single pixel
(or all except the first one are black, then it might also look
as if the loop would be run only once;-)

But what looks strange is the innermost loop. You never tell what
exactly 'f' is but I would tend to assume that it is a file object
for your CSV file, which you opened somewhere before. And now you
read it in completely when dealing with the very first pixel of
your image. Afterwards, when dealing with the other pixels of the
image, there's nothing left to be read in, so the inner loop won't
be run again, making it appear as if the outer loops would only be
run once.

If my assumptions are correct and you want to read in the file
again and again for each pixel then you should either open it
again and again for each pixel or, probably better, reset the
file object so that it "points" back to the start of the file
before the start of the innermost loop, using the seek() method
- a simple "f.seek(0)" should do the job (assuming that this is
a normal file, i.e. one that can be "rewound" and not e.g. a re-
directed pipe).

An even better solution (if you have enough memory) might be to
read in the whole file into a list and iterate over that instead
of the file itself. And better than that might be to build a
dictionary of values in the file that you can use later on, so
you don't have to run over the whole file again and again:

d = { }
for l in f :
sp = split( l, ',' )
if len( sp ) == 14 :
d[ int( sp[ 0 ].strip( ) ) ] = int( sp[ 13 ].strip( ) )

Then you can later check directly if some color value (what
you have named 'reg_num') is in the file by using 

if reg_num in d :

and the corresponding value from the file (what you assign
to 'classification') is simply the value of the dictionary
for the key given by 'reg_num'. i.e.

classification = d[ reg_num ]

Regards, Jens
-- 
  \   Jens Thoms Toerring  ___  j...@toerring.de
   \__  http://toerring.de
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Nested For loop not running full

[inshu chauhan]

, 2013 at 3:15 PM, Chris Angelico  wrote:

> On Fri, Apr 26, 2013 at 7:36 PM, inshu chauhan 
> wrote:
> >
> > On Fri, Apr 26, 2013 at 2:39 PM, Peter Otten <__pete...@web.de> wrote:
> >>
> >> f = open(...)
> >>
> >> in the code you are not showing with
> >>
> >> f == list(open(...))
> >
> > f is just a text file(csv format).. so why list ??
>
> (That should be =, not ==)
>
> Instead of having an open file object, you would instead have a list
> of the lines in the file. That can be iterated over more than once.
>
> >> The reasonable thing to do is of course to move the preprocessing (e.g.
> >> csv-
> >> parsing) out of the sy and sx loops.
> >
> >
> > I did this but again then what I intend to do is not really happening,
> For
> > every pixel I read,  I want to traverse the full file, so that the
> > information I am taking from pixel have to match in one of the line in
> the
> > file. Can this be done by modifying my code ? or something new has to be
> > devised ?
>
> How large is the file? There are two easy solutions:
>
> 1) Open and close the file every time you touch a pixel
> 2) Open the file once, read it all into memory, and then iterate over
> the in-memory copy every pixel
>
> If your file is insanely large then the first option may be better,
> but for anything less than five yottabytes, go with the second. (Okay,
> I may be exaggerating slightly... let's say anything less than half
> your RAM. So if you have 10YB of memory, then I wasn't exaggerating.)
> That's why Peter suggested creating a list; you iterate over the list.
> Another way to do it is to parse the file once and retain a more
> efficient and useful structured form of the data... which is the other
> thing Peter suggested ("move the preprocessing (e.g. csv-
> parsing) out of the sy and sx loops").
>
> So, yeah. Listen to Peter Otten, he knows what he's talking about :)
>
> ChrisA
>

Yes I am trying Peter's way and my file is just 500 KB .. :)
-- 
http://mail.python.org/mailman/listinfo/python-list

New Dropbox alternative with 10GB free space

[akanakshaa]

Everybody, there is a new online storage startup, similar to Dropbox or 
box.com, get your free 10 GB easily, 5GB on signup and 5GB if you use the 
following referral link. 
* hit the referral link https://copy.com?r=WGQBHr 
* create an account (5GB) 
* install client on any device (5GB) 
Have fun! 
-- 
http://mail.python.org/mailman/listinfo/python-list

TypeError: Can't convert 'int' object to str implicitly

[tunacubes]

Hey,

Let me explain what my program is supposed to do...

I am using a macro program in conjunction with a python script I am writing. 
The macro inputs data into a database we use at my job, blah blah blah.

The script asks how many numbers (devices) you need to enter. Next, it asks you 
to input the device numbers. When you input them, it creates a list with all of 
the devices. I then tell it to go into the script of the Macro (firstdev.ahk) 
that will run on the back-end, and replace the word "device" with the first 
device in the list. It then should execute the Macro, change the device number 
back to the word "Device" for future use, and then delete the first number from 
the list. It will repeat as long as there are numbers in the list.

The error I receive is "TypeError: Can't convert 'int' object to str 
implicitly" when it tries to put the device into the macro script. It worked 
fine when I just had it input one device into the script without the use of 
lists, but for whatever reason, using a list does not play nice with replacing 
the words "Device" with the number from the list. Here is my script. I will 
give you up until the part that the error occurs, as everything afterwords is 
pointless.

I am fairly new to python, so if anything looks screwed up or like I am an 
idiot, it is because I am.

import fileinput, sys, os
devlist = []
maxdev = int(input("How many devices to add: "))
curdev = int("0")
while curdev < maxdev:
try:
Number = int(input("Enter Device number: "))
devlist.append(Number)
curdev = curdev + 1
except ValueError:
print("Please enter a valid number")
ready = 0
while ready != "Y" and ready != "y" and ready != "yes" and ready != "YES" and 
ready != "ready" and ready != "Ready":
try:
ready = input("Confirm when you are ready ")
except ValueError:
print("shit broke. ")
##This next step will seek out the word Device within firstdev.ahk, and replace 
with devlist[0]
for line in fileinput.input(["firstdev.ahk"], inplace=True):
line = line.replace("device", devlist[0])
sys.stdout.write(line)
##next step runs firstdev.ahk
os.system('firstdev.ahk')
##next step is replacing devlist[0] with "device" 
for line in fileinput.input(["firstdev.ahk"], inplace=True):
line = line.replace(devlist[0], "device")
sys.stdout.write(line)
del devlist[0] #deleting the first item from the list. next steps will repeat.


Thanks in advance.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[Peter Otten]

tunacu...@gmail.com wrote:

> Hey,
> 
> Let me explain what my program is supposed to do...
> 
> I am using a macro program in conjunction with a python script I am
> writing. The macro inputs data into a database we use at my job, blah blah
> blah.
> 
> The script asks how many numbers (devices) you need to enter. Next, it
> asks you to input the device numbers. When you input them, it creates a
> list with all of the devices. I then tell it to go into the script of the
> Macro (firstdev.ahk) that will run on the back-end, and replace the word
> "device" with the first device in the list. It then should execute the
> Macro, change the device number back to the word "Device" for future use,
> and then delete the first number from the list. It will repeat as long as
> there are numbers in the list.
> 
> The error I receive is "TypeError: Can't convert 'int' object to str
> implicitly" when it tries to put the device into the macro script. 

Python is trying hard to give you a meaningful error message and shows the 
line causing the error in the traceback. It pays to read carefully -- or to 
post it here if it doesn't make sense to you.

> devlist = []
...
> Number = int(input("Enter Device number: "))
> devlist.append(Number)
...
> line = line.replace(devlist[0], "device")

devList is a list of integers, and devlist[0] is thus an int.

>>> "foo device bar\n".replace(42, "device")
Traceback (most recent call last):
  File "", line 1, in 
TypeError: Can't convert 'int' object to str implicitly

Implicitly? So let's try and convert the int /explicitly/ .

>>> "foo device bar\n".replace(str(42), "device")
'foo device bar\n'

No error, but probably still not what you expected. Can you sort it out 
yourself?

> I am fairly new to python, so if anything looks screwed up or like I am an
> idiot, it is because I am.

I like to see a bit of self-deprecation now and then, but on this list 
complete tracebacks find even more love ;)


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[Chris Angelico]

On Sat, Apr 27, 2013 at 12:26 AM,   wrote:
> Hey,
> The error I receive is "TypeError: Can't convert 'int' object to str 
> implicitly" when it tries to put the device into the macro script. It worked 
> fine when I just had it input one device into the script without the use of 
> lists, but for whatever reason, using a list does not play nice with 
> replacing the words "Device" with the number from the list. Here is my 
> script. I will give you up until the part that the error occurs, as 
> everything afterwords is pointless.
>
> I am fairly new to python, so if anything looks screwed up or like I am an 
> idiot, it is because I am.

Welcome to Python. One thing that you'll quickly learn is that the
full traceback is REALLY helpful; in this case, there won't be much
traceback, but at least you'll get the line number. I'm looking at
this line as being the culprit:

> line = line.replace("device", devlist[0])

All you need to do is convert that to a string:
line = line.replace("device", str(devlist[0]))
and that should fix your problem. But while I'm here, may I offer a
few other tips?

(By the way, I'm going to assume you're using Python 3 here, because
otherwise your "confirm when ready" would be bombing. But it helps to
say what version you're using.)

> import fileinput, sys, os
> devlist = []
> maxdev = int(input("How many devices to add: "))
> curdev = int("0")

Just 'curdev = 0' would work here. No need to convert a string to an
integer - just use a literal integer.

> while curdev < maxdev:

UI suggestion: Let the user keep on inputting device numbers until
some kind of sentinel (eg a blank line), unless it makes really good
sense to ask for the number up-front.

> try:
> Number = int(input("Enter Device number: "))
> devlist.append(Number)
> curdev = curdev + 1
> except ValueError:
> print("Please enter a valid number")

You can save the hassle of maintaining curdev by simply looking at
len(devlist) - that is, change your condition to "while len(devlist) <
maxdev".

> ready = 0
> while ready != "Y" and ready != "y" and ready != "yes" and ready != "YES" and 
> ready != "ready" and ready != "Ready":

Here's a really cool tip. You can simply check set membership:

while ready not in {"Y","y","yes","YES","ready","Ready"}:

But I'd be inclined to make the UI a bit tighter here and make it
something like:

ready = input("Confirm when you are ready : ")

and then just look for either "Y" or "y", nothing more. But that's up
to you. You know who's going to use this program, I don't. (The
capitalized N representing that the default is No is a convention you
may or may not want to follow.)

> try:
> ready = input("Confirm when you are ready ")
> except ValueError:
> print("shit broke. ")

Drop the try/except; if there's something wrong (and, btw, I'm not
sure what circumstances would trigger a ValueError here - maybe
someone else knows of something?), just let the exception terminate
the program. There's nothing useful to do here anyway :)

> ##This next step will seek out the word Device within firstdev.ahk, and 
> replace with devlist[0]
> for line in fileinput.input(["firstdev.ahk"], inplace=True):
> line = line.replace("device", devlist[0])
> sys.stdout.write(line)

You're reading a single file with constant name here. Save yourself
some trouble: just open the file directly.

for line in open("firstdev.ahk"):
print(line.replace("device", str(devlist[0])))

Though I'm not sure why you want to write to stdout here. Were you
intending to write back to another file? I'm getting a bit lost in
your code here. My understanding of your intentions, from your
comments, is that you actually want to repeat most of the code from
here on for each device number entered - that strongly suggests a
'for' loop bracketing the whole thing. Is that what you meant this to
do? Currently, it'll get to the 'del' at the bottom, and then just
terminate.

> ##next step runs firstdev.ahk
> os.system('firstdev.ahk')
> ##next step is replacing devlist[0] with "device"
> for line in fileinput.input(["firstdev.ahk"], inplace=True):
> line = line.replace(devlist[0], "device")
> sys.stdout.write(line)

And here you repeat everything from above. Again, not sure what you're
intending to do here.

> del devlist[0] #deleting the first item from the list. next steps will repeat.

The steps won't repeat by themselves, so this is where I'm thinking
you actually want a for loop.

I hope you don't take this the wrong way, as I feel now (on skimming
over this email prior to sending) that I've kinda pulled your code to
pieces a bit! It's just advice, just suggestions; this is your code
and nobody else's. You may very well disagree with any or all of what
I've said, and that's to be expected. It's also a great way to get
discussion going, and discussion is one of the best ways for everyone
to learn :)

Chris Angelico
-- 
http://mail.python.org/mailman/listinf

Re: TypeError: Can't convert 'int' object to str implicitly

[Chris Angelico]

On Sat, Apr 27, 2013 at 12:26 AM,   wrote:
> ##This next step will seek out the word Device within firstdev.ahk, and 
> replace with devlist[0]
> for line in fileinput.input(["firstdev.ahk"], inplace=True):
> line = line.replace("device", devlist[0])
> sys.stdout.write(line)
> ##next step runs firstdev.ahk
> os.system('firstdev.ahk')
> ##next step is replacing devlist[0] with "device"
> for line in fileinput.input(["firstdev.ahk"], inplace=True):
> line = line.replace(devlist[0], "device")
> sys.stdout.write(line)

I've checked out what fileinput.input() is doing here (ought to have
done that earlier, sorry!) and I now understand this block of code
more. You're replacing that word _in the file, on disk_, and then
making the inverse replacement. This strikes me as dangerous; if
anything happens to your process in the middle, the file will be
damaged on disk. I would STRONGLY recommend rewriting this to use some
other file - for instance, a temporary file. I haven't looked into the
details, as I haven't actually done this lately in Python, but you
should be able to use tempfile.NamedTemporaryFile(delete=False) [1],
write to it, make it executable, run it, and unlink it. That way,
you're creating a temporary file to run, not running the original.
This is semantically different from your code, but I think it'd be a
lot safer.

[1] http://docs.python.org/3.3/library/tempfile.html#tempfile.NamedTemporaryFile

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[tunacubes]

On Friday, April 26, 2013 10:53:44 AM UTC-4, Peter Otten wrote:
> tunacu...@gmail.com wrote:
> 
> 
> 
> > Hey,
> 
> > 
> 
> > Let me explain what my program is supposed to do...
> 
> > 
> 
> > I am using a macro program in conjunction with a python script I am
> 
> > writing. The macro inputs data into a database we use at my job, blah blah
> 
> > blah.
> 
> > 
> 
> > The script asks how many numbers (devices) you need to enter. Next, it
> 
> > asks you to input the device numbers. When you input them, it creates a
> 
> > list with all of the devices. I then tell it to go into the script of the
> 
> > Macro (firstdev.ahk) that will run on the back-end, and replace the word
> 
> > "device" with the first device in the list. It then should execute the
> 
> > Macro, change the device number back to the word "Device" for future use,
> 
> > and then delete the first number from the list. It will repeat as long as
> 
> > there are numbers in the list.
> 
> > 
> 
> > The error I receive is "TypeError: Can't convert 'int' object to str
> 
> > implicitly" when it tries to put the device into the macro script. 
> 
> 
> 
> Python is trying hard to give you a meaningful error message and shows the 
> 
> line causing the error in the traceback. It pays to read carefully -- or to 
> 
> post it here if it doesn't make sense to you.
> 
> 
> 
> > devlist = []
> 
> ...
> 
> > Number = int(input("Enter Device number: "))
> 
> > devlist.append(Number)
> 
> ...
> 
> > line = line.replace(devlist[0], "device")
> 
> 
> 
> devList is a list of integers, and devlist[0] is thus an int.
> 
> 
> 
> >>> "foo device bar\n".replace(42, "device")
> 
> Traceback (most recent call last):
> 
>   File "", line 1, in 
> 
> TypeError: Can't convert 'int' object to str implicitly
> 
> 
> 
> Implicitly? So let's try and convert the int /explicitly/ .
> 
> 
> 
> >>> "foo device bar\n".replace(str(42), "device")
> 
> 'foo device bar\n'
> 
> 
> 
> No error, but probably still not what you expected. Can you sort it out 
> 
> yourself?
> 
> 
> 
> > I am fairly new to python, so if anything looks screwed up or like I am an
> 
> > idiot, it is because I am.
> 
> 
> 
> I like to see a bit of self-deprecation now and then, but on this list 
> 
> complete tracebacks find even more love ;)

Thank you, Peter. This was a tremendous help. Got it working. 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[tunacubes]

On Friday, April 26, 2013 11:05:29 AM UTC-4, Chris Angelico wrote:
> On Sat, Apr 27, 2013 at 12:26 AM,   wrote:
> 
> > ##This next step will seek out the word Device within firstdev.ahk, and 
> > replace with devlist[0]
> 
> > for line in fileinput.input(["firstdev.ahk"], inplace=True):
> 
> > line = line.replace("device", devlist[0])
> 
> > sys.stdout.write(line)
> 
> > ##next step runs firstdev.ahk
> 
> > os.system('firstdev.ahk')
> 
> > ##next step is replacing devlist[0] with "device"
> 
> > for line in fileinput.input(["firstdev.ahk"], inplace=True):
> 
> > line = line.replace(devlist[0], "device")
> 
> > sys.stdout.write(line)
> 
> 
> 
> I've checked out what fileinput.input() is doing here (ought to have
> 
> done that earlier, sorry!) and I now understand this block of code
> 
> more. You're replacing that word _in the file, on disk_, and then
> 
> making the inverse replacement. This strikes me as dangerous; if
> 
> anything happens to your process in the middle, the file will be
> 
> damaged on disk. I would STRONGLY recommend rewriting this to use some
> 
> other file - for instance, a temporary file. I haven't looked into the
> 
> details, as I haven't actually done this lately in Python, but you
> 
> should be able to use tempfile.NamedTemporaryFile(delete=False) [1],
> 
> write to it, make it executable, run it, and unlink it. That way,
> 
> you're creating a temporary file to run, not running the original.
> 
> This is semantically different from your code, but I think it'd be a
> 
> lot safer.
> 
> 
> 
> [1] 
> http://docs.python.org/3.3/library/tempfile.html#tempfile.NamedTemporaryFile
> 
> 
> 
> ChrisA

Thank you, Chris! I got it working and am going to take your advice on the 
tempfile idea. I actually ran into the problem you were referring to, and kept 
the .ahk files backed up elsewhere for when this situation arose. I appreciate 
the help!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[Chris Angelico]

On Sat, Apr 27, 2013 at 1:22 AM,   wrote:
>> I've checked out what fileinput.input() is doing here (ought to have
>>
>> done that earlier, sorry!) and I now understand this block of code
>>
>> more. You're replacing that word _in the file, on disk_, and then
>>
>> making the inverse replacement. This strikes me as dangerous; if
>>
>> anything happens to your process in the middle, the file will be
>>
>> damaged on disk. I would STRONGLY recommend rewriting this to use some
>>
>> other file - for instance, a temporary file. I haven't looked into the
>>
>> details, as I haven't actually done this lately in Python, but you
>>
>> should be able to use tempfile.NamedTemporaryFile(delete=False) [1],
>>
>> write to it, make it executable, run it, and unlink it. That way,
>>
>> you're creating a temporary file to run, not running the original.
>>
>> This is semantically different from your code, but I think it'd be a
>>
>> lot safer.
>
> Thank you, Chris! I got it working and am going to take your advice on the 
> tempfile idea. I actually ran into the problem you were referring to, and 
> kept the .ahk files backed up elsewhere for when this situation arose. I 
> appreciate the help!

Awesome!

Hey, you want to know an easy way to show your appreciation for the
people who answer your questions? It's really simple:

Step 1: Don't use Google Groups to post.
Step 2: Profit!

Have a look at the quoted text in your responses. See how it's
double-spaced and really obnoxious? That's not your fault, it's
because Google Groups is buggy. There are plenty of other ways to read
and post; I personally have subscribed to the mailing list and read it
using Gmail, and there are a variety of other ways too. If you _must_
use Google Groups, check out this page for some recommendations on how
to not offend people.

http://wiki.python.org/moin/GoogleGroupsPython

There are a number of non-G-rated terms going around that fairly
accurately describe what Google Groups posts look like, but since I
prefer milder language, I'll just repeat the word "obnoxious", as it
carries all the meaning I need.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python-ideas] Automatic context managers

[Chris Angelico]

On Sat, Apr 27, 2013 at 1:54 AM, MRAB  wrote:
> On 26/04/2013 14:02, anatoly techtonik wrote:
>> This circular reference problem is interesting. In object space it
>> probably looks like a stellar detached from the visible (attached)
>> universe. Is the main problem in detecting it?
>>
> The problem is in knowing in which order the objects should be
> collected.
>
> For example, if A refers to B and B refers to A, should you collect A
> then B, or B then A? If you collect A first, then, for a time, B will
> be referring to a non-existent object. That's not good if the objects
> have destructors which need to be run.

Spin-off thread from python-ideas to discuss a more general question
of garbage collection of cyclic structures.

Once it's been proven that there's an unreferenced cycle, why not
simply dispose of one of the objects, and replace all references to it
(probably only one - preferably pick an object with the fewest
references) with a special temporary object? In fact, that could
probably be done in CPython by wiping out the object in memory and
replacing it with a special marker of some sort, which would then
automatically "take over" all references to the old object. Any
attempt to manipulate this object could simply pop back with a
DestructedObject exception or something.

Is this a plausible (never mind viable yet, just conceptually
plausible) alternative to sticking them into gc.garbage and ignoring
them? It'd allow a double-linked list/tree to function cleanly -
imagine, for instance, something like the DOM facilities available to
web browser scripts:

class DOMObject:
def __init__(self,parent):
self.parent=parent
self.firstchild=self.sibling=None
if not parent: return
if not parent.firstchild:
parent.firstchild=self
else:
child=parent.firstchild
while child.sibling:
child=child.sibling
child.sibling=self
def __del__(self):
print("Disposing of id #%d"%id(self))

document=DOMObject(None)
body=DOMObject(document)
p=DOMObject(body)
p=DOMObject(body)
p=DOMObject(body)
del document,body,p
gc.collect()

The __del__ method would need to clean up the external resources used
by this object, but wouldn't have to walk the tree. Yet, just because
there is a reference loop and there are __del__ methods, the garbage
collector gives up and leaves it to the program author to deal with.

I can understand if this is considered too complicated and too unusual
a circumstance to be worth bothering to support, but I'm curious as to
whether it's at least conceptually reasonable to do something like
this.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Chris Angelico]

[[ Resending under a more appropriate subject line... sorry about
that, ignore the other one as it'll only confuse matters ]]

On Sat, Apr 27, 2013 at 1:54 AM, MRAB  wrote:
> On 26/04/2013 14:02, anatoly techtonik wrote:
>> This circular reference problem is interesting. In object space it
>> probably looks like a stellar detached from the visible (attached)
>> universe. Is the main problem in detecting it?
>>
> The problem is in knowing in which order the objects should be
> collected.
>
> For example, if A refers to B and B refers to A, should you collect A
> then B, or B then A? If you collect A first, then, for a time, B will
> be referring to a non-existent object. That's not good if the objects
> have destructors which need to be run.

Spin-off thread from python-ideas to discuss a more general question
of garbage collection of cyclic structures.

Once it's been proven that there's an unreferenced cycle, why not
simply dispose of one of the objects, and replace all references to it
(probably only one - preferably pick an object with the fewest
references) with a special temporary object? In fact, that could
probably be done in CPython by wiping out the object in memory and
replacing it with a special marker of some sort, which would then
automatically "take over" all references to the old object. Any
attempt to manipulate this object could simply pop back with a
DestructedObject exception or something.

Is this a plausible (never mind viable yet, just conceptually
plausible) alternative to sticking them into gc.garbage and ignoring
them? It'd allow a double-linked list/tree to function cleanly -
imagine, for instance, something like the DOM facilities available to
web browser scripts:

class DOMObject:
def __init__(self,parent):
self.parent=parent
self.firstchild=self.sibling=None
if not parent: return
if not parent.firstchild:
parent.firstchild=self
else:
child=parent.firstchild
while child.sibling:
child=child.sibling
child.sibling=self
def __del__(self):
print("Disposing of id #%d"%id(self))

document=DOMObject(None)
body=DOMObject(document)
p=DOMObject(body)
p=DOMObject(body)
p=DOMObject(body)
del document,body,p
gc.collect()

The __del__ method would need to clean up the external resources used
by this object, but wouldn't have to walk the tree. Yet, just because
there is a reference loop and there are __del__ methods, the garbage
collector gives up and leaves it to the program author to deal with.

I can understand if this is considered too complicated and too unusual
a circumstance to be worth bothering to support, but I'm curious as to
whether it's at least conceptually reasonable to do something like
this.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[Dave Angel]

On 04/26/2013 11:05 AM, Chris Angelico wrote:

On Sat, Apr 27, 2013 at 12:26 AM,   wrote:

##This next step will seek out the word Device within firstdev.ahk, and replace 
with devlist[0]
for line in fileinput.input(["firstdev.ahk"], inplace=True):
 line = line.replace("device", devlist[0])
 sys.stdout.write(line)
##next step runs firstdev.ahk
os.system('firstdev.ahk')
##next step is replacing devlist[0] with "device"
for line in fileinput.input(["firstdev.ahk"], inplace=True):
 line = line.replace(devlist[0], "device")
 sys.stdout.write(line)


I've checked out what fileinput.input() is doing here (ought to have
done that earlier, sorry!) and I now understand this block of code
more. You're replacing that word _in the file, on disk_, and then
making the inverse replacement. This strikes me as dangerous; if
anything happens to your process in the middle, the file will be
damaged on disk. I would STRONGLY recommend rewriting this to use some
other file - for instance, a temporary file. I haven't looked into the
details, as I haven't actually done this lately in Python, but you
should be able to use tempfile.NamedTemporaryFile(delete=False) [1],
write to it, make it executable, run it, and unlink it. That way,
you're creating a temporary file to run, not running the original.
This is semantically different from your code, but I think it'd be a
lot safer.

[1] http://docs.python.org/3.3/library/tempfile.html#tempfile.NamedTemporaryFile

ChrisA



fileinput.Fileinput class already creates the temp file when you specify 
 inplace=True


If it didn't, I'd also have to point out the hazards of doing in-place 
updates in a text file where the new data and old is a different length.


There still may be reasons to make an explicit backup, but I don't know 
what they are.



--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[Chris Angelico]

On Sat, Apr 27, 2013 at 3:23 AM, Dave Angel  wrote:
> On 04/26/2013 11:05 AM, Chris Angelico wrote:
>> I've checked out what fileinput.input() is doing here (ought to have
>> done that earlier, sorry!) and I now understand this block of code
>> more. You're replacing that word _in the file, on disk_, and then
>> making the inverse replacement. This strikes me as dangerous; if
>> anything happens to your process in the middle, the file will be
>> damaged on disk. I would STRONGLY recommend rewriting this to use some
>> other file - for instance, a temporary file.
>
> fileinput.Fileinput class already creates the temp file when you specify
> inplace=True
>
> If it didn't, I'd also have to point out the hazards of doing in-place
> updates in a text file where the new data and old is a different length.
>
> There still may be reasons to make an explicit backup, but I don't know what
> they are.

That's true if something goes wrong during the actual writing of the
file only. But if the process bombs in the middle of the execution
phase (which in the Python script is a single os.system() call), then
the file will have been one-way changed on the disk - hence,
"damaged". The explicit temporary file (and executing the temp file)
is a much safer way to do it.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Dave Angel]

On 04/26/2013 12:54 PM, Chris Angelico wrote:

[[ Resending under a more appropriate subject line... sorry about
that, ignore the other one as it'll only confuse matters ]]

On Sat, Apr 27, 2013 at 1:54 AM, MRAB  wrote:

On 26/04/2013 14:02, anatoly techtonik wrote:

This circular reference problem is interesting. In object space it
probably looks like a stellar detached from the visible (attached)
universe. Is the main problem in detecting it?


The problem is in knowing in which order the objects should be
collected.

For example, if A refers to B and B refers to A, should you collect A
then B, or B then A? If you collect A first, then, for a time, B will
be referring to a non-existent object. That's not good if the objects
have destructors which need to be run.


Spin-off thread from python-ideas to discuss a more general question
of garbage collection of cyclic structures.

Once it's been proven that there's an unreferenced cycle, why not
simply dispose of one of the objects, and replace all references to it
(probably only one - preferably pick an object with the fewest
references) with a special temporary object? In fact, that could
probably be done in CPython by wiping out the object in memory and
replacing it with a special marker of some sort, which would then
automatically "take over" all references to the old object. Any
attempt to manipulate this object could simply pop back with a
DestructedObject exception or something.

Is this a plausible (never mind viable yet, just conceptually
plausible) alternative to sticking them into gc.garbage and ignoring
them? It'd allow a double-linked list/tree to function cleanly -
imagine, for instance, something like the DOM facilities available to
web browser scripts:

class DOMObject:
 def __init__(self,parent):
 self.parent=parent
 self.firstchild=self.sibling=None
 if not parent: return
 if not parent.firstchild:
 parent.firstchild=self
 else:
 child=parent.firstchild
 while child.sibling:
 child=child.sibling
 child.sibling=self
 def __del__(self):
 print("Disposing of id #%d"%id(self))

document=DOMObject(None)
body=DOMObject(document)
p=DOMObject(body)
p=DOMObject(body)
p=DOMObject(body)
del document,body,p
gc.collect()

The __del__ method would need to clean up the external resources used
by this object, but wouldn't have to walk the tree. Yet, just because
there is a reference loop and there are __del__ methods, the garbage
collector gives up and leaves it to the program author to deal with.

I can understand if this is considered too complicated and too unusual
a circumstance to be worth bothering to support, but I'm curious as to
whether it's at least conceptually reasonable to do something like
this.

ChrisA



I don't see what your "special" temporary object actually accomplishes. 
 Seems to me you need to declare that your __del__() methods promise 
not to reference each other, and the gc would then check all objects in 
the cycle, and do its present behavior if any of the destructors is not 
specially declared.


I'm not sure how often you'd have a non-trivial destructor that wouldn't 
reference any objects.  And doing a static analysis of what will happen 
during the destructor would be pretty messy.  So the best I and come up 
with is to keep the declaration, but require a try/catch to cleanly 
terminate each destructor if it ever references anything in the tree.


--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[Dave Angel]

On 04/26/2013 01:28 PM, Chris Angelico wrote:

On Sat, Apr 27, 2013 at 3:23 AM, Dave Angel  wrote:

On 04/26/2013 11:05 AM, Chris Angelico wrote:

I've checked out what fileinput.input() is doing here (ought to have
done that earlier, sorry!) and I now understand this block of code
more. You're replacing that word _in the file, on disk_, and then
making the inverse replacement. This strikes me as dangerous; if
anything happens to your process in the middle, the file will be
damaged on disk. I would STRONGLY recommend rewriting this to use some
other file - for instance, a temporary file.


fileinput.Fileinput class already creates the temp file when you specify
inplace=True

If it didn't, I'd also have to point out the hazards of doing in-place
updates in a text file where the new data and old is a different length.

There still may be reasons to make an explicit backup, but I don't know what
they are.


That's true if something goes wrong during the actual writing of the
file only. But if the process bombs in the middle of the execution
phase (which in the Python script is a single os.system() call), then
the file will have been one-way changed on the disk - hence,
"damaged". The explicit temporary file (and executing the temp file)
is a much safer way to do it.



Only two of those sentences makes sense to me. I have no idea what 
"execution phase" means, and don't know what would be done via an 
os.system() call.  I don't know what "one-way change" means


If I were specifying the fileinput stuff, I'd have said the new data 
should be written to the temp file, so that at no time was the original 
file in an in-between state.


--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Chris Angelico]

On Sat, Apr 27, 2013 at 3:42 AM, Dave Angel  wrote:
> I don't see what your "special" temporary object actually accomplishes.
> Seems to me you need to declare that your __del__() methods promise not to
> reference each other, and the gc would then check all objects in the cycle,
> and do its present behavior if any of the destructors is not specially
> declared.

It wouldn't be declared; it'd simply throw an exception if anything
different happened.

> I'm not sure how often you'd have a non-trivial destructor that wouldn't
> reference any objects.  And doing a static analysis of what will happen
> during the destructor would be pretty messy.  So the best I and come up with
> is to keep the declaration, but require a try/catch to cleanly terminate
> each destructor if it ever references anything in the tree.

And yeah. If you catch the exception inside __del__, you can cope with
the destructed object yourself (or LBLY, if you wish). Alternatively,
you just proceed as normal, and when your __del__ throws an exception,
the gc then copes (not sure *how* it should cope - log it to stderr
and carry on?). Same as normal exception handling.

The advantage of this style is that the code to deal with the cycle is
kept right in the cyclic object's destructor - right where the problem
is. Doing it through gc.garbage requires that some other operation
periodically check for garbage - after the GC has done its own
periodic check. Seems simpler/cleaner to do it as part of the gc run
itself.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[Chris Angelico]

On Sat, Apr 27, 2013 at 3:52 AM, Dave Angel  wrote:
> On 04/26/2013 01:28 PM, Chris Angelico wrote:
>>
>> On Sat, Apr 27, 2013 at 3:23 AM, Dave Angel  wrote:
>>>
>>> On 04/26/2013 11:05 AM, Chris Angelico wrote:

 I've checked out what fileinput.input() is doing here (ought to have
 done that earlier, sorry!) and I now understand this block of code
 more. You're replacing that word _in the file, on disk_, and then
 making the inverse replacement. This strikes me as dangerous; if
 anything happens to your process in the middle, the file will be
 damaged on disk. I would STRONGLY recommend rewriting this to use some
 other file - for instance, a temporary file.
>>>
>>>
>>> fileinput.Fileinput class already creates the temp file when you specify
>>> inplace=True
>>>
>>> If it didn't, I'd also have to point out the hazards of doing in-place
>>> updates in a text file where the new data and old is a different length.
>>>
>>> There still may be reasons to make an explicit backup, but I don't know
>>> what
>>> they are.
>>
>>
>> That's true if something goes wrong during the actual writing of the
>> file only. But if the process bombs in the middle of the execution
>> phase (which in the Python script is a single os.system() call), then
>> the file will have been one-way changed on the disk - hence,
>> "damaged". The explicit temporary file (and executing the temp file)
>> is a much safer way to do it.
>>
>
> Only two of those sentences makes sense to me. I have no idea what
> "execution phase" means, and don't know what would be done via an
> os.system() call.  I don't know what "one-way change" means
>
> If I were specifying the fileinput stuff, I'd have said the new data should
> be written to the temp file, so that at no time was the original file in an
> in-between state.

Here's a massive simplification of the OP's program:

1. Build a list of device IDs
2. Edit the file "firstdev.ahk" and replace all instances of "device"
with the device ID
3. Execute the now-edited firstdev.ahk using os.system()
4. Reverse the edit of firstdev.ahk, replacing all instances of the
device ID with the word "device".

Apart from the risk of accidentally changing back something that
wasn't changed in the first place (which the OP may know to be
impossible, eg if the macro file has no numbers in it), this has the
risk that a computer failure in step 3 will leave the file on disk in
its edited state. That's what I'm concerned about. By writing the
modified .ahk content to a different file and then executing the other
file, he would avoid editing the template at all. It'd also then be
multi-process safe, for what that's worth.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [pyxl] Re: xlrd 0.9.2 released!

[Chris Withers]

Hi Ondrej,

I don't know what a Fedora/EPEL update is.

If you're going to have two versions, I would suggest:

0.7.9 - for backwards compatibility with Python 2.5 and less, and older 
0.6 and 0.7 xlrd releases.


0.9.2 - for Python 3 support, and with the latest and greatest features.

cheers,

Chris

On 23/04/2013 12:24, Ondrej Ján wrote:

Hello. Can you please tell me, how compatible is this version with older
versions? In Fedora/CentOS we have versions 0.7 and 0.6. Can I release
and Fedora/EPEL update to 0.9.2?

Thank you.

SAL

Dňa utorok, 9. apríla 2013 21:38:30 UTC+2 Chris Withers napísal(-a):

Hi All,

I'm pleased to announce the release of xlrd 0.9.2:

http://pypi.python.org/pypi/xlrd/0.9.2


This release includes the following changes:

- Fix some packaging issues that meant docs and examples were missing
from the tarball.

- Fixed a small but serious regression that caused problems opening
.xlsx files.

If you find any problems, please ask about them on the
python...@googlegroups.com  list, or submit an issue on
GitHub:

https://github.com/python-excel/xlrd/issues


Full details of all things Python and Excel related can be found here:

http://www.python-excel.org/

cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk

--
You received this message because you are subscribed to the Google
Groups "python-excel" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to python-excel+unsubscr...@googlegroups.com.
To post to this group, send an email to python-ex...@googlegroups.com.
Visit this group at http://groups.google.com/group/python-excel?hl=en-GB.
For more options, visit https://groups.google.com/groups/opt_out.



__
This email has been scanned by the Symantec Email Security.cloud service.
For more information please visit http://www.symanteccloud.com
__


--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
--
http://mail.python.org/mailman/listinfo/python-list

Re: QTableWidget updating columns in a single row

[Sara Lochtie]

On Wednesday, April 24, 2013 11:51:04 AM UTC-7, Vincent Vande Vyvre wrote:
Thanks. I think I should be able to figure out from here. I appreciate all of 
the help!



> Le 24/04/2013 19:12, Sara Lochtie a écrit :
> 
> > On Tuesday, April 23, 2013 11:22:29 PM UTC-7, Sara Lochtie wrote:
> 
> >> I have written a GUI that gets data sent to it in real time and this data 
> >> is displayed in a table. Every time data is sent in it is displayed in the 
> >> table in a new row. My problem is that I would like to have the data just 
> >> replace the old in the first row.
> 
> >>
> 
> >>
> 
> >>
> 
> >> The table has 6 columns (A, B, C, D, E, F) I want the new data to continue 
> >> replacing the old data in the same row unless the data that goes under 
> >> column A changes, at which point a new row would be added.
> 
> >>
> 
> >>
> 
> >>
> 
> >> Does anyone have tips on how to approach this? I can post a portion of my 
> >> code to get a better idea of what I have done.
> 
> > So that is where I am stuck. I don't how to compare them and I am trying to 
> > avoiding saving the data to a file.
> 
> >
> 
> > This is the code that I have:
> 
> >
> 
> >
> 
> >
> 
> >
> 
> >
> 
> > if msg.arg2() != ERROR:
> 
> > entry = (A, B, C, D, E, F)  
> 
> > self.data.append(entry)
> 
> > 
> 
> >  data = self.data
> 
> >  
> 
> >  # Display how many runs occurred
> 
> >  self.statusBar().showMessage('Data read. %s Run(s) Occurred.' % 
> > self.runCount)
> 
> >  
> 
> >  # Populates table by adding only new entries to the end of the 
> > table
> 
> >  lastRow = self.table.rowCount()
> 
> >  self.table.setRowCount(len(data))
> 
> >  for entryPos in range(lastRow, len(data)):
> 
> >  for fieldPos in range(6):
> 
> >  item = 
> > QtGui.QTableWidgetItem(str(data[entryPos][fieldPos]))
> 
> >  self.table.setItem(entryPos, fieldPos, item)
> 
> >  self.table.resizeColumnsToContents()
> 
> >  self.table.horizontalHeader().setStretchLastSection(True)  
> 
> >  self.currentRunLabel.setText('Current Run: ' + str(self.runCount))
> 
> >  self.currentLineLabel.setText('Number of lines: ' + 
> > str(len(self.data)))
> 
> > print('End of %s. run:%s. entries found' % (self.runCount, 
> > len(self.data)))
> 
> As sayed by Chris "Kwpolska", you can compare the new data with the data 
> 
> of the first row.
> 
> 
> 
> Something like that:
> 
> 
> 
>   # Get the content of row 0, column A
> 
> first = str(self.table.item(0, 0).text())
> 
> 
> 
> for entryPos in range(lastRow, len(data)):
> 
>  if str(data[entryPos][0]) == first:
> 
>  self.update_first_row(data[entryPos])
> 
> 
> 
>  else:
> 
>  self.add_new_row(data[entryPos])
> 
> -- 
> 
> Vincent V.V.
> 
> Oqapy  . Qarte 
> 
>  . PaQager 

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: Can't convert 'int' object to str implicitly

[Dave Angel]

On 04/26/2013 02:02 PM, Chris Angelico wrote:




  


Here's a massive simplification of the OP's program:

1. Build a list of device IDs
2. Edit the file "firstdev.ahk" and replace all instances of "device"
with the device ID
3. Execute the now-edited firstdev.ahk using os.system()
4. Reverse the edit of firstdev.ahk, replacing all instances of the
device ID with the word "device".

Apart from the risk of accidentally changing back something that
wasn't changed in the first place (which the OP may know to be
impossible, eg if the macro file has no numbers in it), this has the
risk that a computer failure in step 3 will leave the file on disk in
its edited state. That's what I'm concerned about. By writing the
modified .ahk content to a different file and then executing the other
file, he would avoid editing the template at all. It'd also then be
multi-process safe, for what that's worth.

OK, thanks.  That makes perfect sense.  Somehow I missed that it was the 
altered file that was being used as a script.  I just assumed it was 
like a config file for some program, where the name is hardwired.





--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Dave Angel]

On 04/26/2013 01:57 PM, Chris Angelico wrote:

On Sat, Apr 27, 2013 at 3:42 AM, Dave Angel  wrote:

I don't see what your "special" temporary object actually accomplishes.
Seems to me you need to declare that your __del__() methods promise not to
reference each other, and the gc would then check all objects in the cycle,
and do its present behavior if any of the destructors is not specially
declared.


It wouldn't be declared; it'd simply throw an exception if anything
different happened.


I'm not sure how often you'd have a non-trivial destructor that wouldn't
reference any objects.  And doing a static analysis of what will happen
during the destructor would be pretty messy.  So the best I and come up with
is to keep the declaration, but require a try/catch to cleanly terminate
each destructor if it ever references anything in the tree.


And yeah. If you catch the exception inside __del__, you can cope with
the destructed object yourself (or LBLY, if you wish). Alternatively,
you just proceed as normal, and when your __del__ throws an exception,
the gc then copes (not sure *how* it should cope - log it to stderr
and carry on?). Same as normal exception handling.

The advantage of this style is that the code to deal with the cycle is
kept right in the cyclic object's destructor - right where the problem
is. Doing it through gc.garbage requires that some other operation
periodically check for garbage - after the GC has done its own
periodic check. Seems simpler/cleaner to do it as part of the gc run
itself.



You must think me dense by now.  But I don't understand what the two 
different garbage collection operations are that you're positing.


As far as I know, there's ref counting, which is quick, and frees 
something as soon as the count goes to zero.  Then there's gc, which has 
to scan through all the objects from a known starting set, and identify 
those things which aren't accessible, and free any that don't have a 
__del__() method.


And it's only in the gc step that cycles and such are identifiable.


--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: [pyxl] Re: xlrd 0.9.2 released!

[Ondrej Ján]

Hello,

  Fedora is an Linux distribution maintained by Red Hat.
(www.fedoraproject.org). EPEL are packages fro Red Hat Enterprise Linux
and CentOS Linux distribution. As an maintainer I am creating and updating
packages available for these distributions.

  If users have python-xlrd installed, package maintainer should release
only updates, which are API compatible with older versions.

  Both Fedora and CentOS has python>=2.6, python3 is available for Fedora
too (no plans to release python3-xlrd now).

  So can I release an update for all my Linux distributions with backward
compatibility of this package? xlrd-0.8 was requested for xlsx support.

  Are there some howtos describing upgrading applications using xlrd
to newer versions? These documents can help me to choose, if I can release
an update.

  Thank you

SAL

On Fri, Apr 26, 2013 at 06:47:08PM +0100, Chris Withers wrote:
> Hi Ondrej,
> 
> I don't know what a Fedora/EPEL update is.
> 
> If you're going to have two versions, I would suggest:
> 
> 0.7.9 - for backwards compatibility with Python 2.5 and less, and
> older 0.6 and 0.7 xlrd releases.
> 
> 0.9.2 - for Python 3 support, and with the latest and greatest features.
> 
> cheers,
> 
> Chris
> 
> On 23/04/2013 12:24, Ondrej Ján wrote:
> >Hello. Can you please tell me, how compatible is this version with older
> >versions? In Fedora/CentOS we have versions 0.7 and 0.6. Can I release
> >and Fedora/EPEL update to 0.9.2?
> >
> >Thank you.
> >
> >SAL
> >
> >Dňa utorok, 9. apríla 2013 21:38:30 UTC+2 Chris Withers napísal(-a):
> >
> >Hi All,
> >
> >I'm pleased to announce the release of xlrd 0.9.2:
> >
> >http://pypi.python.org/pypi/xlrd/0.9.2
> >
> >
> >This release includes the following changes:
> >
> >- Fix some packaging issues that meant docs and examples were missing
> >from the tarball.
> >
> >- Fixed a small but serious regression that caused problems opening
> >.xlsx files.
> >
> >If you find any problems, please ask about them on the
> >python...@googlegroups.com  list, or submit an issue on
> >GitHub:
> >
> >https://github.com/python-excel/xlrd/issues
> >
> >
> >Full details of all things Python and Excel related can be found here:
> >
> >http://www.python-excel.org/
> >
> >cheers,
> >
> >Chris
> >
> >--
> >Simplistix - Content Management, Batch Processing & Python Consulting
> >- http://www.simplistix.co.uk
> >
> >--
> >You received this message because you are subscribed to the Google
> >Groups "python-excel" group.
> >To unsubscribe from this group and stop receiving emails from it, send
> >an email to python-excel+unsubscr...@googlegroups.com.
> >To post to this group, send an email to python-ex...@googlegroups.com.
> >Visit this group at http://groups.google.com/group/python-excel?hl=en-GB.
> >For more options, visit https://groups.google.com/groups/opt_out.
> >
> >
> >
> >__
> >This email has been scanned by the Symantec Email Security.cloud service.
> >For more information please visit http://www.symanteccloud.com
> >__
> 
> -- 
> Simplistix - Content Management, Batch Processing & Python Consulting
> - http://www.simplistix.co.uk
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Python-ideas] Automatic context managers

[88888 Dihedral]

Chris Angelico於 2013年4月27日星期六UTC+8上午12時52分38秒寫道：
> On Sat, Apr 27, 2013 at 1:54 AM, MRAB  wrote:
> 
> > On 26/04/2013 14:02, anatoly techtonik wrote:
> 
> >> This circular reference problem is interesting. In object space it
> 
> >> probably looks like a stellar detached from the visible (attached)
> 
> >> universe. Is the main problem in detecting it?
> 
> >>
> 
> > The problem is in knowing in which order the objects should be
> 
> > collected.
> 
> >
> 
> > For example, if A refers to B and B refers to A, should you collect A
> 
> > then B, or B then A? If you collect A first, then, for a time, B will
> 
> > be referring to a non-existent object. That's not good if the objects
> 
> > have destructors which need to be run.
> 
> 
> 
> Spin-off thread from python-ideas to discuss a more general question
> 
> of garbage collection of cyclic structures.
> 
> 
> 
> Once it's been proven that there's an unreferenced cycle, why not
> 
> simply dispose of one of the objects, and replace all references to it
> 
> (probably only one - preferably pick an object with the fewest
> 
> references) with a special temporary object? In fact, that could
> 
> probably be done in CPython by wiping out the object in memory and
> 
> replacing it with a special marker of some sort, which would then
> 
> automatically "take over" all references to the old object. Any
> 
> attempt to manipulate this object could simply pop back with a
> 
> DestructedObject exception or something.
> 
> 
> 
> Is this a plausible (never mind viable yet, just conceptually
> 
> plausible) alternative to sticking them into gc.garbage and ignoring
> 
> them? It'd allow a double-linked list/tree to function cleanly -
> 
> imagine, for instance, something like the DOM facilities available to
> 
> web browser scripts:
> 
> 
> 
> class DOMObject:
> 
>   def __init__(self,parent):
> 
>   self.parent=parent
> 
>   self.firstchild=self.sibling=None
> 
>   if not parent: return
> 
>   if not parent.firstchild:
> 
>   parent.firstchild=self
> 
>   else:
> 
>   child=parent.firstchild
> 
>   while child.sibling:
> 
>   child=child.sibling
> 
>   child.sibling=self
> 
>   def __del__(self):
> 
>   print("Disposing of id #%d"%id(self))
> 
> 
> 
> document=DOMObject(None)
> 
> body=DOMObject(document)
> 
> p=DOMObject(body)
> 
> p=DOMObject(body)
> 
> p=DOMObject(body)
> 
> del document,body,p
> 
> gc.collect()
> 
> 
> 
> The __del__ method would need to clean up the external resources used
> 
> by this object, but wouldn't have to walk the tree. Yet, just because
> 
> there is a reference loop and there are __del__ methods, the garbage
> 
> collector gives up and leaves it to the program author to deal with.
> 
> 
> 
> I can understand if this is considered too complicated and too unusual
> 
> a circumstance to be worth bothering to support, but I'm curious as to
> 
> whether it's at least conceptually reasonable to do something like
> 
> this.
> 
> 
> 
> ChrisA

Please use the deep-copy methods  from time to time to disentangle
referenced objects in python.

The cyclic reference cycle has to be broken by some mean first
in python to proceed for further actions in the gc.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Ian Kelly]

On Fri, Apr 26, 2013 at 10:54 AM, Chris Angelico  wrote:
> Once it's been proven that there's an unreferenced cycle, why not
> simply dispose of one of the objects, and replace all references to it
> (probably only one - preferably pick an object with the fewest
> references) with a special temporary object? In fact, that could
> probably be done in CPython by wiping out the object in memory and
> replacing it with a special marker of some sort, which would then
> automatically "take over" all references to the old object. Any
> attempt to manipulate this object could simply pop back with a
> DestructedObject exception or something.

I think it still boils down to the same problem -- how should Python
*predictably* choose which object will be disposed of in order to
break the cycle?  I don't see that this question is any different than
asking how should Python choose which __del__ method should be called
first, since the object so disposed of would still need its __del__
method called.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Ian Kelly]

On Fri, Apr 26, 2013 at 1:31 PM, Dave Angel  wrote:
> On 04/26/2013 01:57 PM, Chris Angelico wrote:
>> And yeah. If you catch the exception inside __del__, you can cope with
>> the destructed object yourself (or LBLY, if you wish). Alternatively,
>> you just proceed as normal, and when your __del__ throws an exception,
>> the gc then copes (not sure *how* it should cope - log it to stderr
>> and carry on?). Same as normal exception handling.
>>
>> The advantage of this style is that the code to deal with the cycle is
>> kept right in the cyclic object's destructor - right where the problem
>> is. Doing it through gc.garbage requires that some other operation
>> periodically check for garbage - after the GC has done its own
>> periodic check. Seems simpler/cleaner to do it as part of the gc run
>> itself.
>>
>
> You must think me dense by now.  But I don't understand what the two
> different garbage collection operations are that you're positing.
>
> As far as I know, there's ref counting, which is quick, and frees something
> as soon as the count goes to zero.  Then there's gc, which has to scan
> through all the objects from a known starting set, and identify those things
> which aren't accessible, and free any that don't have a __del__() method.
>
> And it's only in the gc step that cycles and such are identifiable.

Whenever the GC finds a cycle that is unreferenced but uncollectable,
it stores those objects in the list gc.garbage.  At that point, if the
user wishes to clean up those cycles, it is up to them to delve into
gc.garbage, untangle the objects contained within, break the cycles,
and remove them from the list so that they can be freed by the ref
counter.  This user-supplied step is what Chris is referring to as
"some other" periodic check.  If the user does not do this, then those
objects simply remain in the gc.garbage list until the program
terminates.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Dave Angel]

On 04/26/2013 06:43 PM, Ian Kelly wrote:


   


Whenever the GC finds a cycle that is unreferenced but uncollectable,
it stores those objects in the list gc.garbage.  At that point, if the
user wishes to clean up those cycles, it is up to them to delve into
gc.garbage, untangle the objects contained within, break the cycles,
and remove them from the list so that they can be freed by the ref
counter.  This user-supplied step is what Chris is referring to as
"some other" periodic check.  If the user does not do this, then those
objects simply remain in the gc.garbage list until the program
terminates.



I didn't know there was a callback that a user could hook into.  That's 
very interesting.


--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Dave Angel]

On 04/26/2013 06:50 PM, Ian Kelly wrote:

On Fri, Apr 26, 2013 at 10:54 AM, Chris Angelico  wrote:

Once it's been proven that there's an unreferenced cycle, why not
simply dispose of one of the objects, and replace all references to it
(probably only one - preferably pick an object with the fewest
references) with a special temporary object? In fact, that could
probably be done in CPython by wiping out the object in memory and
replacing it with a special marker of some sort, which would then
automatically "take over" all references to the old object. Any
attempt to manipulate this object could simply pop back with a
DestructedObject exception or something.


I think it still boils down to the same problem -- how should Python
*predictably* choose which object will be disposed of in order to
break the cycle?  I don't see that this question is any different than
asking how should Python choose which __del__ method should be called
first, since the object so disposed of would still need its __del__
method called.



Perhaps if the __del__ methods disposed of the non-cyclic parts first, 
an exception doing the cyclic cleanups wouldn't hurt.  Picture a 
doubly-linked list of file-type objects.  Each one could release its 
file before trying to do anything with the links.


--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Skip Montanaro]

> Whenever the GC finds a cycle that is unreferenced but uncollectable,
> it stores those objects in the list gc.garbage.  At that point, if the
> user wishes to clean up those cycles, it is up to them to delve into
> gc.garbage, untangle the objects contained within, break the cycles,
> and remove them from the list so that they can be freed by the ref
> counter.

I wonder if it would be useful to provide a gc.garbagehook analogous
to sys.excepthook?
Users could assign a function of their choice to much the cyclic
garbage periodically.

Just a thought, flying out of my fingers before my brain could stop it...

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Chris Angelico]

On Sat, Apr 27, 2013 at 8:50 AM, Ian Kelly  wrote:
> On Fri, Apr 26, 2013 at 10:54 AM, Chris Angelico  wrote:
>> Once it's been proven that there's an unreferenced cycle, why not
>> simply dispose of one of the objects, and replace all references to it
>> (probably only one - preferably pick an object with the fewest
>> references) with a special temporary object? In fact, that could
>> probably be done in CPython by wiping out the object in memory and
>> replacing it with a special marker of some sort, which would then
>> automatically "take over" all references to the old object. Any
>> attempt to manipulate this object could simply pop back with a
>> DestructedObject exception or something.
>
> I think it still boils down to the same problem -- how should Python
> *predictably* choose which object will be disposed of in order to
> break the cycle?  I don't see that this question is any different than
> asking how should Python choose which __del__ method should be called
> first, since the object so disposed of would still need its __del__
> method called.

It wouldn't need to. It just chooses arbitrarily. Maybe it picks the
one with the lowest id(), maybe it picks the object that takes up the
most space, maybe the one that would have been put into gc.garbage
first. But it's got to die tomorrow, so it really doesn't matter[1],
and there's no way for the GC to know which one won't be referenced by
the other's __del__ method.

[1] I'm involved in a Gilbert and Sullivan Society performance,
curtain up in less than three hours, so I'm morally obligated to quote
G&S somewhere.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Chris Angelico]

On Sat, Apr 27, 2013 at 9:45 AM, Dave Angel  wrote:
> I didn't know there was a callback that a user could hook into.  That's very
> interesting.
>

On Sat, Apr 27, 2013 at 10:22 AM, Skip Montanaro  wrote:
>> Whenever the GC finds a cycle that is unreferenced but uncollectable,
>> it stores those objects in the list gc.garbage.  At that point, if the
>> user wishes to clean up those cycles, it is up to them to delve into
>> gc.garbage, untangle the objects contained within, break the cycles,
>> and remove them from the list so that they can be freed by the ref
>> counter.
>
> I wonder if it would be useful to provide a gc.garbagehook analogous
> to sys.excepthook?
> Users could assign a function of their choice to much the cyclic
> garbage periodically.
>
> Just a thought, flying out of my fingers before my brain could stop it...

As far as I know, Dave, there isn't currently one; Skip, that's close
to what I'm talking about - it saves on the periodic check. But
burying it in gc.garbagehook implies having a separate piece of code
that knows how to break the reference cycles, whereas the __del__
method puts the code right there in the code that has the problem.
Actually, *ANY* solution to this problem implies having __del__ able
to cope with the cycle being broken. Here's an example, perhaps a
silly one, but not far different in nature from some things I've done
in C++. (Granted, all the Python implementations of those same
algorithms have involved built-in types rather than linked lists, but
still.)

class DLCircList:
def __init__(self,payload):
self.payload=payload
self.next=self.prev=self
print("Creating node: %s"%self.payload)
def __del__(self):
print("Deleting node %s from cycle 
%s"%(self.payload,self.enum()))
self.prev.next=self.next
self.next.prev=self.prev
def attach(self,other):
assert(self.next==self) # Don't attach twice
self.prev=other
self.next=other.next
other.next=self
self.next.prev=self
print("Adding node %s to cycle %s"%(self.payload,self.enum()))
def enum(self):
"""Return a list of all node payloads in this cycle."""
ptr=self.next
nodes=[self.payload]
while ptr!=self:
nodes.append(ptr.payload)
ptr=ptr.next
return nodes

lst=DLCircList("foo")
DLCircList("bar").attach(lst)
DLCircList("quux").attach(lst)
DLCircList("asdf").attach(lst)
DLCircList("qwer").attach(lst)
DLCircList("zxcv").attach(lst)
print("Enumerating list: %s"%lst.enum())

del lst
import gc
gbg=gc.collect()
print("And we have garbage: %s"%gbg)
print(gc.garbage)



Supposing you did this many many times, and you wanted decent garbage
collection. How would you write a __del__ method, how would you write
something to clean up gc.garbage? One way or another, something will
have to deal with the possibility that the invariants have been
broken, so my theory is that that possibility should be entirely
within __del__. (Since __del__ calls enum(), it's possible for enum()
to throw DestructedObject or whatever, but standard exception handling
will deal with that.)

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Skip Montanaro]

>From the Zen of Python:

> In the face of ambiguity, refuse the temptation to guess.

I believe the reason something isn't already done to break cycles is
that the authors of the cyclic garbage collector considered the above
aphorism.  They rely on the author of the code with the cycles to
figure out how to break them.  All I was suggesting was that Python
could provide a hook where the programmer could codify his algorithm
for breaking cycles.

Skip
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: CPython's cyclic garbage collector (was [Python-ideas] Automatic context managers)

[Chris Angelico]

On Sat, Apr 27, 2013 at 12:12 PM, Skip Montanaro  wrote:
> From the Zen of Python:
>
>> In the face of ambiguity, refuse the temptation to guess.
>
> I believe the reason something isn't already done to break cycles is
> that the authors of the cyclic garbage collector considered the above
> aphorism.  They rely on the author of the code with the cycles to
> figure out how to break them.  All I was suggesting was that Python
> could provide a hook where the programmer could codify his algorithm
> for breaking cycles.

Sure, and that makes good sense. The hook would be an improvement
(maybe it gets passed a list of garbage that's about to be added to
gc.garbage, but will be re-checked for cycles after the hook, so all
you have to do is break the cycle in the hook and it'll work), but the
implication is that an external piece of code knows about the possible
cycles and how to deal with them. I'd really rather have something
right there in the class. Effectively, something like this:

whenever gc.garbage has content:
turn_into(gc.garbage[0],None) # or a special "I am destroyed" object
del gc.garbage[:]

It doesn't matter which one gets taken out. Yes, I suppose that's a
form of guessing, but whatever rules get put in place - whether by the
language or by your own script - it'll always be possible to conjure a
scenario where tossing a coin is the only way to pick which one goes.

ChrisA
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: baffled classes within a function namespace. Evaluation order.

[Steven D'Aprano]

On Fri, 26 Apr 2013 09:15:51 +0200, Peter Otten wrote:

> Define a global variable x and run it again. I don't have an ancient
> Python lying around but my "prediction" is
> 
 x = 42
 f().x
> 42
> 
> which would be the same behaviour as that of Python 3.3.


/facepalm


You're absolutely right. Sorry for the noise.




-- 
Steven
-- 
http://mail.python.org/mailman/listinfo/python-list

File Read issue by using module binascii

[Jimmie He]

When I run the readbmp on an example.bmp(about 100k),the Shell is become to "No 
respose",when I change f.read() to f.read(1000),it is ok,could someone tell me 
the excat reason for this?
Thank you in advance!

Python Code as below!!

import binascii

def read_bmp():
f = open('example.bmp','rb')
rawdata = f.read()   #f.read(1000) is ok
hexstr = binascii.b2a_hex(rawdata)   #Get an HEX number
bsstr = bin (int(hexstr,16))[2:]
f.close()
print('bin: ',bsstr,type(bsstr))
return
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: File Read issue by using module binascii

[Jimmie He]

when I commet the line of "print('bin: ',bsstr,type(bsstr)) ",it can be run,so 
maybe the problem is the memory allocate of so long strings..Am I right?

在 2013年4月27日星期六UTC+8上午11时57分45秒，Jimmie He写道：
> When I run the readbmp on an example.bmp(about 100k),the Shell is become to 
> "No respose",when I change f.read() to f.read(1000),it is ok,could someone 
> tell me the excat reason for this?
> 
> Thank you in advance!
> 
> 
> 
> Python Code as below!!
> 
> 
> 
> import binascii
> 
> 
> 
> def read_bmp():
> 
> f = open('example.bmp','rb')
> 
> rawdata = f.read()   #f.read(1000) is ok
> 
> hexstr = binascii.b2a_hex(rawdata)   #Get an HEX number
> 
> bsstr = bin (int(hexstr,16))[2:]
> 
> f.close()
> 
> print('bin: ',bsstr,type(bsstr))
> 
> return
-- 
http://mail.python.org/mailman/listinfo/python-list