RE: To clarify how Python handles two equal objects

[Jen Kris via Python-list]

Avi,

Thanks for your comments.  You make a good point. 

Going back to my original question, and using your slice() example: 

middle_by_two = slice(5, 10, 2)
nums = [n for n in range(12)]
q = nums[middle_by_two]
x = id(q)
b = q
y = id(b)

If I assign "b" to "q", then x and y match – they point to the same memory 
until "b" OR "q" are  reassigned to something else.  If "q" changes during the 
lifetime of "b" then it’s not safe to use the pointer to "q" for "b", as in:

nums = [n for n in range(2, 14)]
q = nums[middle_by_two]
x = id(q)
y = id(b)

Now "x" and "y" are different, as we would expect.  So when writing a spot 
speed up in a compiled language, you can see in the Python source if either is 
reassigned, so you’ll know how to handle it.  The motivation behind my question 
was that in a compiled extension it’s faster to borrow a pointer than to move 
an entire array if it’s possible, but special care must be taken. 

Jen



Jan 12, 2023, 20:51 by avi.e.gr...@gmail.com:

> Jen,
>
> It is dangerous territory you are treading as there are times all or parts of 
> objects are copied, or changed in place or the method you use to make a view 
> is not doing quite what you want.
>
> As an example, you can create a named slice such as:
>
>  middle_by_two = slice(5, 10, 2)
>
> The above is not in any sense pointing at anything yet. But given a long 
> enough list or other such objects, it will take items (starting at index 0) 
> starting with item that are at indices 5 then 7 then 9  as in this:
>
>  nums = [n for n in range(12)]
>  nums[middle_by_two]
>
> [5, 7, 9]
>
> The same slice will work on anything else:
>
>  list('abcdefghijklmnopqrstuvwxyz')[middle_by_two]
> ['f', 'h', 'j']
>
> So although you may think the slice is bound to something, it is not. It is 
> an object that only later is briefly connected to whatever you want to apply 
> it to.
>
> If I later change nums, above, like this:
>
>  nums = [-3, -2, -1] + nums
>  nums
> [-3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
>  nums[middle_by_two]
> [2, 4, 6]
>
> In the example, you can forget about whether we are talking about pointers 
> directly or indirectly or variable names and so on. Your "view" remains valid 
> ONLY as long as you do not change either the slice or the underlying object 
> you are applying to -- at least not the items you want to extract.
>
> Since my example inserted three new items at the start using negative numbers 
> for illustration, you would need to adjust the slice by making a new slice 
> designed to fit your new data. The example below created an adjusted slice 
> that adds 3 to the start and stop settings of the previous slice while 
> copying the step value and then it works on the elongated object:
>
>  middle_by_two_adj = slice(middle_by_two.start + 3, middle_by_two.stop + 3, 
> middle_by_two.step)
>  nums[middle_by_two_adj]
> [5, 7, 9]
>
> A suggestion is  that whenever you are not absolutely sure that the contents 
> of some data structure might change without your participation, then don't 
> depend on various kinds of aliases to keep the contents synchronized. Make a 
> copy, perhaps  a deep copy and make sure the only thing ever changing it is 
> your code and later, if needed, copy the result back to any other data 
> structure. Of course, if anything else is accessing the result in the 
> original in between, it won't work.
>
> Just FYI, a similar analysis applies to uses of the numpy and pandas and 
> other modules if you get some kind of object holding indices to a series such 
> as integers or Booleans and then later try using it after the number of items 
> or rows or columns have changed. Your indices no longer match.
>
> Avi
>
> -Original Message-
> From: Python-list  On 
> Behalf Of Jen Kris via Python-list
> Sent: Wednesday, January 11, 2023 1:29 PM
> To: Roel Schroeven 
> Cc: python-list@python.org
> Subject: Re: To clarify how Python handles two equal objects
>
> Thanks for your comments.  After all, I asked for clarity so it’s not 
> pedantic to be precise, and you’re helping to clarify. 
>
> Going back to my original post,
>
> mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ]
> arr1 = mx1[2]
>
> Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed 
> because while they are different names, they are the assigned same memory 
> location (pointer).  Similarly, if I write "mx1[2][1] += 5" then again both 
> names will be updated. 
>
> That’s what I meant by "an operation on one is an operation on the other."  
> To be more precise, an operation on one name will be reflected in the other 
> name.  The difference is in the names,  not the pointers.  Each name has the 
> same pointer in my example, but operations can be done in Python using either 
> name. 
>
>
>
>
> Jan 11, 2023, 09:13 by r...@roelschroeven.net:
>
>> Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list:
>>
>>> Yes, I did understand that.  In your example, "a" and "b" are the same 
>>> pointer, so a

Re: To clarify how Python handles two equal objects

[Bob van der Poel]

It seems to me that the the entire concept of relying on python's idea of
where an object is stored is just plain dangerous. A most simple example
might be:
   >>> a=1
   >>> b=1
   >>> a is b
  True
  >>> a=1234
  >>> b=1234
  >>> a is b
  False

Not sure what happens if you manipulate the data referenced by 'b' in the
first example thinking you are changing something referred to by 'a' ...
but you might be smart to NOT think that you know.



On Fri, Jan 13, 2023 at 9:00 AM Jen Kris via Python-list <
python-list@python.org> wrote:

>
> Avi,
>
> Thanks for your comments.  You make a good point.
>
> Going back to my original question, and using your slice() example:
>
> middle_by_two = slice(5, 10, 2)
> nums = [n for n in range(12)]
> q = nums[middle_by_two]
> x = id(q)
> b = q
> y = id(b)
>
> If I assign "b" to "q", then x and y match – they point to the same memory
> until "b" OR "q" are  reassigned to something else.  If "q" changes during
> the lifetime of "b" then it’s not safe to use the pointer to "q" for "b",
> as in:
>
> nums = [n for n in range(2, 14)]
> q = nums[middle_by_two]
> x = id(q)
> y = id(b)
>
> Now "x" and "y" are different, as we would expect.  So when writing a spot
> speed up in a compiled language, you can see in the Python source if either
> is reassigned, so you’ll know how to handle it.  The motivation behind my
> question was that in a compiled extension it’s faster to borrow a pointer
> than to move an entire array if it’s possible, but special care must be
> taken.
>
> Jen
>
>
>
> Jan 12, 2023, 20:51 by avi.e.gr...@gmail.com:
>
> > Jen,
> >
> > It is dangerous territory you are treading as there are times all or
> parts of objects are copied, or changed in place or the method you use to
> make a view is not doing quite what you want.
> >
> > As an example, you can create a named slice such as:
> >
> >  middle_by_two = slice(5, 10, 2)
> >
> > The above is not in any sense pointing at anything yet. But given a long
> enough list or other such objects, it will take items (starting at index 0)
> starting with item that are at indices 5 then 7 then 9  as in this:
> >
> >  nums = [n for n in range(12)]
> >  nums[middle_by_two]
> >
> > [5, 7, 9]
> >
> > The same slice will work on anything else:
> >
> >  list('abcdefghijklmnopqrstuvwxyz')[middle_by_two]
> > ['f', 'h', 'j']
> >
> > So although you may think the slice is bound to something, it is not. It
> is an object that only later is briefly connected to whatever you want to
> apply it to.
> >
> > If I later change nums, above, like this:
> >
> >  nums = [-3, -2, -1] + nums
> >  nums
> > [-3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
> >  nums[middle_by_two]
> > [2, 4, 6]
> >
> > In the example, you can forget about whether we are talking about
> pointers directly or indirectly or variable names and so on. Your "view"
> remains valid ONLY as long as you do not change either the slice or the
> underlying object you are applying to -- at least not the items you want to
> extract.
> >
> > Since my example inserted three new items at the start using negative
> numbers for illustration, you would need to adjust the slice by making a
> new slice designed to fit your new data. The example below created an
> adjusted slice that adds 3 to the start and stop settings of the previous
> slice while copying the step value and then it works on the elongated
> object:
> >
> >  middle_by_two_adj = slice(middle_by_two.start + 3, middle_by_two.stop +
> 3, middle_by_two.step)
> >  nums[middle_by_two_adj]
> > [5, 7, 9]
> >
> > A suggestion is  that whenever you are not absolutely sure that the
> contents of some data structure might change without your participation,
> then don't depend on various kinds of aliases to keep the contents
> synchronized. Make a copy, perhaps  a deep copy and make sure the only
> thing ever changing it is your code and later, if needed, copy the result
> back to any other data structure. Of course, if anything else is accessing
> the result in the original in between, it won't work.
> >
> > Just FYI, a similar analysis applies to uses of the numpy and pandas and
> other modules if you get some kind of object holding indices to a series
> such as integers or Booleans and then later try using it after the number
> of items or rows or columns have changed. Your indices no longer match.
> >
> > Avi
> >
> > -Original Message-
> > From: Python-list 
> On Behalf Of Jen Kris via Python-list
> > Sent: Wednesday, January 11, 2023 1:29 PM
> > To: Roel Schroeven 
> > Cc: python-list@python.org
> > Subject: Re: To clarify how Python handles two equal objects
> >
> > Thanks for your comments.  After all, I asked for clarity so it’s not
> pedantic to be precise, and you’re helping to clarify.
> >
> > Going back to my original post,
> >
> > mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ]
> > arr1 = mx1[2]
> >
> > Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be
> changed because while they are different name

Re: To clarify how Python handles two equal objects

[Axel Reichert]

 writes:

> As an example, you can create a named slice such as:
>
>   middle_by_two = slice(5, 10, 2)
>
> The above is not in any sense pointing at anything yet.

>From a functional programming point of view this just looks like a
partially applied function, and with this in mind the behaviour to me
seems to be completely as expected. No surprises here, or do I miss
something?

Best regards

Axel
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Zen of D.E.K.

[Ethan Furman]

On 1/13/23 09:06, Stefan Ram wrote:

>"Beautiful is better than ugly." - The Zen of Python
>
>This says nothing. You have to sacrifice something that
>really has /value/!
>
>"[A]esthetics are more important than efficiency." - Donald E. Knuth

[okay, falling for the troll bait]

Those two things do not say the same thing; further, in Python at least, and depending on the situation, aesthetics may 
/not/ be more important than efficiency.


--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list

Re: To clarify how Python handles two equal objects

[Jen Kris via Python-list]

Bob, 

Your examples show a and b separately defined.  My example is where the 
definition is a=1; b = a.  But I'm only interested in arrays.  I would not rely 
on this for integers, and there's not likely to be any real cost savings there. 
  


Jan 13, 2023, 08:45 by b...@mellowood.ca:

> It seems to me that the the entire concept of relying on python's idea of 
> where an object is stored is just plain dangerous. A most simple example 
> might be:
>    >>> a=1
>    >>> b=1
>    >>> a is b
>   True
>   >>> a=1234
>   >>> b=1234
>   >>> a is b
>   False
>
> Not sure what happens if you manipulate the data referenced by 'b' in the 
> first example thinking you are changing something referred to by 'a' ... but 
> you might be smart to NOT think that you know.
>
>
>
> On Fri, Jan 13, 2023 at 9:00 AM Jen Kris via Python-list <> 
> python-list@python.org> > wrote:
>
>>
>> Avi,
>>  
>>  Thanks for your comments.  You make a good point. 
>>  
>>  Going back to my original question, and using your slice() example: 
>>  
>>  middle_by_two = slice(5, 10, 2)
>>  nums = [n for n in range(12)]
>>  q = nums[middle_by_two]
>>  x = id(q)
>>  b = q
>>  y = id(b)
>>  
>>  If I assign "b" to "q", then x and y match – they point to the same memory 
>> until "b" OR "q" are  reassigned to something else.  If "q" changes during 
>> the lifetime of "b" then it’s not safe to use the pointer to "q" for "b", as 
>> in:
>>  
>>  nums = [n for n in range(2, 14)]
>>  q = nums[middle_by_two]
>>  x = id(q)
>>  y = id(b)
>>  
>>  Now "x" and "y" are different, as we would expect.  So when writing a spot 
>> speed up in a compiled language, you can see in the Python source if either 
>> is reassigned, so you’ll know how to handle it.  The motivation behind my 
>> question was that in a compiled extension it’s faster to borrow a pointer 
>> than to move an entire array if it’s possible, but special care must be 
>> taken. 
>>  
>>  Jen
>>  
>>  
>>  
>>  Jan 12, 2023, 20:51 by >> avi.e.gr...@gmail.com>> :
>>  
>>  > Jen,
>>  >
>>  > It is dangerous territory you are treading as there are times all or 
>> parts of objects are copied, or changed in place or the method you use to 
>> make a view is not doing quite what you want.
>>  >
>>  > As an example, you can create a named slice such as:
>>  >
>>  >  middle_by_two = slice(5, 10, 2)
>>  >
>>  > The above is not in any sense pointing at anything yet. But given a long 
>> enough list or other such objects, it will take items (starting at index 0) 
>> starting with item that are at indices 5 then 7 then 9  as in this:
>>  >
>>  >  nums = [n for n in range(12)]
>>  >  nums[middle_by_two]
>>  >
>>  > [5, 7, 9]
>>  >
>>  > The same slice will work on anything else:
>>  >
>>  >  list('abcdefghijklmnopqrstuvwxyz')[middle_by_two]
>>  > ['f', 'h', 'j']
>>  >
>>  > So although you may think the slice is bound to something, it is not. It 
>> is an object that only later is briefly connected to whatever you want to 
>> apply it to.
>>  >
>>  > If I later change nums, above, like this:
>>  >
>>  >  nums = [-3, -2, -1] + nums
>>  >  nums
>>  > [-3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
>>  >  nums[middle_by_two]
>>  > [2, 4, 6]
>>  >
>>  > In the example, you can forget about whether we are talking about 
>> pointers directly or indirectly or variable names and so on. Your "view" 
>> remains valid ONLY as long as you do not change either the slice or the 
>> underlying object you are applying to -- at least not the items you want to 
>> extract.
>>  >
>>  > Since my example inserted three new items at the start using negative 
>> numbers for illustration, you would need to adjust the slice by making a new 
>> slice designed to fit your new data. The example below created an adjusted 
>> slice that adds 3 to the start and stop settings of the previous slice while 
>> copying the step value and then it works on the elongated object:
>>  >
>>  >  middle_by_two_adj = slice(middle_by_two.start + 3, middle_by_two.stop + 
>> 3, middle_by_two.step)
>>  >  nums[middle_by_two_adj]
>>  > [5, 7, 9]
>>  >
>>  > A suggestion is  that whenever you are not absolutely sure that the 
>> contents of some data structure might change without your participation, 
>> then don't depend on various kinds of aliases to keep the contents 
>> synchronized. Make a copy, perhaps  a deep copy and make sure the only thing 
>> ever changing it is your code and later, if needed, copy the result back to 
>> any other data structure. Of course, if anything else is accessing the 
>> result in the original in between, it won't work.
>>  >
>>  > Just FYI, a similar analysis applies to uses of the numpy and pandas and 
>> other modules if you get some kind of object holding indices to a series 
>> such as integers or Booleans and then later try using it after the number of 
>> items or rows or columns have changed. Your indices no longer match.
>>  >
>>  > Avi
>>  >
>>  > -Original Message-
>>  > From: Python-list > 
>> gmail..

Re: To clarify how Python handles two equal objects

[Peter J. Holzer]

On 2023-01-13 16:57:45 +0100, Jen Kris via Python-list wrote:
> Thanks for your comments.  You make a good point. 
> 
> Going back to my original question, and using your slice() example: 
> 
> middle_by_two = slice(5, 10, 2)
> nums = [n for n in range(12)]
> q = nums[middle_by_two]
> x = id(q)
> b = q
> y = id(b)
> 
> If I assign "b" to "q",

You don't asssign b to q. You assign q to b. Assignment is not
commutative, the direction matters.

> then x and y match – they point to the same memory until "b" OR "q"
> are  reassigned to something else.

Correct.

>  If "q" changes during the lifetime of "b" then it’s not safe to use
>the pointer to "q" for "b", as in:

There is no pointer to q[1]. q is a pointer to something (an object of type
list with 3 elements).

b is a pointer to the same object at this point. Of course, if you
assign a different pointer to q, then q will point to the new object
from that point on while b will continue to point to the original object
(until it is also re-assigned).

> nums = [n for n in range(2, 14)]
> q = nums[middle_by_two]
> x = id(q)
> y = id(b)
> 
> Now "x" and "y" are different, as we would expect.
> So when writing a spot speed up in a compiled language,

If you are writing a Python compiler you certainly will have to adjust
for the fact that the same variable may hold pointers to very different
objects over its lifetime. But as far as I understand it, you aren't
trying to write a Python compiler, just an extension, so that shouldn't
concern you.

> The motivation behind my question was that in a compiled extension
> it’s faster to borrow a pointer than to move an entire array if it’s
> possible, but special care must be taken. 

I still don't understand what you mean by that.

hp

[1] Not quite true. The run-time system must keep track of the
variables, so there likely is a pointer to q somewhere. But it's (IMHO)
irrelevant to this discussion, unless the extension is trying to look up
variables by name or something like that.

-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: The Zen of D.E.K.

[Peter J. Holzer]

On 2023-01-13 10:00:01 -0800, Ethan Furman wrote:
> On 1/13/23 09:06, Stefan Ram wrote:
> >"Beautiful is better than ugly." - The Zen of Python
> >
> >This says nothing. You have to sacrifice something that
> >really has /value/!

Time?

Making something beautiful takes time.


> >"[A]esthetics are more important than efficiency." - Donald E. Knuth
> 
> [okay, falling for the troll bait]
> 
> Those two things do not say the same thing; further, in Python at least, and
> depending on the situation, aesthetics may /not/ be more important than
> efficiency.

This is true independent of language.

Aesthetics /may/ be more important than efficiency.
Aesthetics /may/ not be more important than efficiency.

It depends on the application.

Although I would deposit that if you care about run-time efficiency (as
opposed to developer-time efficiency), Python might not be your language
of choice at least for the part where efficiency is crucial.

hp


-- 
   _  | Peter J. Holzer| Story must make more sense than reality.
|_|_) ||
| |   | h...@hjp.at |-- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |   challenge!"


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

RE: To clarify how Python handles two equal objects

[avi.e.gross]

Jen,

This may not be on target but I was wondering about your needs in this 
category. Are all your data in a form where all in a cluster are the same 
object type, such as floating point?

Python has features designed to allow you to get multiple views on such objects 
such as memoryview that can be used to say see an array as a matrix of n rows 
by m columns, or m x n, or any other combo. And of course the fuller numpy 
package has quite a few features.

However, as you note, there is no guarantee that any reference to the data may 
not shift away from it unless you build fairly convoluted logic or data 
structures such as having an object that arranges to do something when you try 
to remove it, such as tinkering with the __del__ method as well as whatever 
method is used to try to set it to a new value. I guess that might make sense 
for something like asynchronous programming including when setting locks so 
multiple things cannot overlap when being done.

Anyway, some of the packages like numpy are optimized in many ways but if you 
want to pass a subset of sorts to make processing faster, I suspect you could 
do things like pass a memoryview but it might not be faster than what you build 
albeit probably more reliable and portable.

I note another odd idea that others may have mentioned, with caution.

If you load the sys module, you can CAREFULLY use code like this.

a="Something Unique"
sys.getrefcount(a)
2

Note if a==1 you will get some huge number of references and this is 
meaningless. The 2 above is because asking about how many references also 
references it.

So save what ever number you have and see what happens when you make a second 
reference or a third, and what happens if you delete or alter a reference:

a="Something Unique"
sys.getrefcount(a)
2
b = a
sys.getrefcount(a)
3
sys.getrefcount(b)
3
c = b
d = a
sys.getrefcount(a)
5
sys.getrefcount(d)
5
del(a)
sys.getrefcount(d)
4
b = "something else"
sys.getrefcount(d)
3

So, in theory, you could carefully write your code to CHECK the reference count 
had not changed but there remain edge cases where a removed reference is 
replaced by yet another new reference and you would have no idea.

Avi


-Original Message-
From: Python-list  On 
Behalf Of Jen Kris via Python-list
Sent: Wednesday, January 11, 2023 1:29 PM
To: Roel Schroeven 
Cc: python-list@python.org
Subject: Re: To clarify how Python handles two equal objects

Thanks for your comments.  After all, I asked for clarity so it’s not pedantic 
to be precise, and you’re helping to clarify.  

Going back to my original post,

mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ]
arr1 = mx1[2]

Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed 
because while they are different names, they are the assigned same memory 
location (pointer).  Similarly, if I write "mx1[2][1] += 5" then again both 
names will be updated. 

That’s what I meant by "an operation on one is an operation on the other."  To 
be more precise, an operation on one name will be reflected in the other name.  
The difference is in the names,  not the pointers.  Each name has the same 
pointer in my example, but operations can be done in Python using either name. 




Jan 11, 2023, 09:13 by r...@roelschroeven.net:

> Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list:
>
>> Yes, I did understand that.  In your example, "a" and "b" are the same 
>> pointer, so an operation on one is an operation on the other (because 
>> they’re the same memory block).
>>
>
> Sorry if you feel I'm being overly pedantic, but your explanation "an 
> operation on one is an operation on the other (because they’re the same 
> memory block)" still feels a bit misguided. "One" and "other" still make it 
> sound like there are two objects, and "an operation on one" and "an operation 
> on the other" make it sound like there are two operations.
> Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or 
> convenience, sometimes we really need to be precise. I think this is a case 
> where we need to be precise.
>
> So, to be precise: there is only one object, with possible multiple names to 
> it. We can change the object, using one of the names. That is one and only 
> one operation on one and only one object. Since the different names refer to 
> the same object, that change will of course be visible through all of them.
> Note that 'name' in that sentence doesn't just refer to variables (mx1, arr1, 
> ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), loop 
> variables, function arguments.
>
> The correct mental model is important here, and I do think you're on track or 
> very close to it, but the way you phrase things does give me that nagging 
> feeling that you still might be just a bit off.
>
> -- 
> "Peace cannot be kept by force. It can only be achieved through 
> understanding."
>  -- Albert Einstein
>
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>

-- 
https

RE: To clarify how Python handles two equal objects

[avi.e.gross]

Jen,

 

Can a compiler, or spot compiler, always know if something that was a reference 
is changed?

 

Obvious examples may be if a change happens in non-deterministic ways such as 
in a fork chosen at random or from user input but also sometimes levels of 
indirection such as deleting an object that internally contains a reference the 
other, perhaps even more indirectly.

 

I know programmers often have their code overhauled and their assumptions go 
away such as someone deciding to create a variable name in an inner scope and 
thus hiding the same variable they were using in an outer scope. So even if 
your code is currently valid, after it changes, a later compiler if it detected 
some change might not want to do your speedup. 

 

I think the subject line of the message we keep exchanging is now a bit 
misleading. It is not about two objects nor really about how python handles 
them. There seem to be one object and possibly multiple views of it and you may 
not want to pass the entire object around or manipulate it a certain way. I am 
not so certain your methods necessarily speed things up as certain views simply 
do calculations on the many places they need to read or change to supply what 
you want.

 

From: Jen Kris  
Sent: Friday, January 13, 2023 10:58 AM
To: avi.e.gr...@gmail.com
Cc: python-list@python.org
Subject: RE: To clarify how Python handles two equal objects

 

 

Avi,

 

Thanks for your comments.  You make a good point. 

 

Going back to my original question, and using your slice() example: 

 

middle_by_two = slice(5, 10, 2)

nums = [n for n in range(12)]

q = nums[middle_by_two]

x = id(q)

b = q

y = id(b)

 

If I assign "b" to "q", then x and y match – they point to the same memory 
until "b" OR "q" are  reassigned to something else.  If "q" changes during the 
lifetime of "b" then it’s not safe to use the pointer to "q" for "b", as in:

 

nums = [n for n in range(2, 14)]

q = nums[middle_by_two]

x = id(q)

y = id(b)

 

Now "x" and "y" are different, as we would expect.  So when writing a spot 
speed up in a compiled language, you can see in the Python source if either is 
reassigned, so you’ll know how to handle it.  The motivation behind my question 
was that in a compiled extension it’s faster to borrow a pointer than to move 
an entire array if it’s possible, but special care must be taken. 

 

Jen

 

 

 

Jan 12, 2023, 20:51 by avi.e.gr...@gmail.com  :

Jen,

 

It is dangerous territory you are treading as there are times all or parts of 
objects are copied, or changed in place or the method you use to make a view is 
not doing quite what you want.

 

As an example, you can create a named slice such as:

 

middle_by_two = slice(5, 10, 2)

 

The above is not in any sense pointing at anything yet. But given a long enough 
list or other such objects, it will take items (starting at index 0) starting 
with item that are at indices 5 then 7 then 9 as in this:

 

nums = [n for n in range(12)]

nums[middle_by_two]

 

[5, 7, 9]

 

The same slice will work on anything else:

 

list('abcdefghijklmnopqrstuvwxyz')[middle_by_two]

['f', 'h', 'j']

 

So although you may think the slice is bound to something, it is not. It is an 
object that only later is briefly connected to whatever you want to apply it to.

 

If I later change nums, above, like this:

 

nums = [-3, -2, -1] + nums

nums

[-3, -2, -1, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]

nums[middle_by_two]

[2, 4, 6]

 

In the example, you can forget about whether we are talking about pointers 
directly or indirectly or variable names and so on. Your "view" remains valid 
ONLY as long as you do not change either the slice or the underlying object you 
are applying to -- at least not the items you want to extract.

 

Since my example inserted three new items at the start using negative numbers 
for illustration, you would need to adjust the slice by making a new slice 
designed to fit your new data. The example below created an adjusted slice that 
adds 3 to the start and stop settings of the previous slice while copying the 
step value and then it works on the elongated object:

 

middle_by_two_adj = slice(middle_by_two.start + 3, middle_by_two.stop + 3, 
middle_by_two.step)

nums[middle_by_two_adj]

[5, 7, 9]

 

A suggestion is that whenever you are not absolutely sure that the contents of 
some data structure might change without your participation, then don't depend 
on various kinds of aliases to keep the contents synchronized. Make a copy, 
perhaps a deep copy and make sure the only thing ever changing it is your code 
and later, if needed, copy the result back to any other data structure. Of 
course, if anything else is accessing the result in the original in between, it 
won't work.

 

Just FYI, a similar analysis applies to uses of the numpy and pandas and other 
modules if you get some kind of object holding indices to a series such as 
integers or Booleans and then l

RE: To clarify how Python handles two equal objects

[avi.e.gross]

Axel and others,

I can appreciate the comparison to a partially applied function but not in
this case. Not that it matters, but this example is more like creating an
object in something like machine learning and initializing parameters
without adding data. Only when you ad data and call upon some transforms and
so on, does it do something.

This case is even more general. You create an object that does NOTHING. It
simply holds a start/end/step set of up to three values. Lots of other
functions will take this object as an argument. It can be used and reused
any number of times. Strictly speaking, code like name[5:10:1] just creates
a transient slice object and then uses that to get the answer. It is not
delayed or partial as much as making one does nothing. 

Stefan mentioned functools.partial and that does create a bit of a curried
function that wraps the data and holds on to it so invoking it sort of wakes
the function up, with some or all data already accessible. A slice does not
do that and needs some other functionality to use IT alongside whatever
object you want to see a slice of.

No special behavior was intended by me. I was illustrating how some methods
of providing a selected view of an object are equally sensitive to the
underlying data changing. A partially applied function that still takes an
argument later, would have a similar problem if underlying data outside the
what is stored within the function, changed, or if the saved was a reference
to something that changed.

But this is really far from unique. In the example given of creating a
partial call, what if you made a second copy to that call then the first
variable to the partial function was re-defined. 

-Original Message-
From: Python-list  On
Behalf Of Axel Reichert
Sent: Friday, January 13, 2023 3:22 AM
To: python-list@python.org
Subject: Re: To clarify how Python handles two equal objects

 writes:

> As an example, you can create a named slice such as:
>
>   middle_by_two = slice(5, 10, 2)
>
> The above is not in any sense pointing at anything yet.

>From a functional programming point of view this just looks like a partially
applied function, and with this in mind the behaviour to me seems to be
completely as expected. No surprises here, or do I miss something?

Best regards

Axel
--
https://mail.python.org/mailman/listinfo/python-list

-- 
https://mail.python.org/mailman/listinfo/python-list