Re: Immutable object thread-safety

[Alcari The Mad]

k3xji wrote:
> Hi all,
> 
> This will probably be a long question/short answer, sorry, but I have
> wandered net about the subject and really feel cannot find just enough
> information.I want to ask my question by giving an example to
> explicitly express my understanding which may be also wrong:
> 
> So, let's have a string in a module like:
> st = 'IAmAstring'

When you make an assignment like that, python generates bytecode that
looks like this(straight from dis.dis):
  1   0 LOAD_CONST   0 ('IAmAstring')
  3 STORE_NAME   0 (a)

That's 1 instruction to load the constant and another one to bind it.

> My understanding: Python allocates a string object and binds the st to
> that string's reference.
> 
> and let's have two threads simply doing following operations on the st
> string:
> 
> Thread A:
> st += 'ThreadAWasHere'
> 
> Thread B
> st = 'ThreadBWasHere'
> 
> Let's think about the following situation:
> Thread A reads the st value, and it is currently binded to an object
> which is holding the string 'IAmAString'.
> So let Thread B starts execution at that point. Note that Thread A
> does not add the new string to object, it only reads st object. So,
> then Thread B just writes to st, and all of a sudden st is binded to
> something which is ThreadBWasHere. And then let's continue Thread A,
> it already readed st's reference which is a reference to 'IamAString'.
> 
> So, finally if the execution flow is like above, we will have
> 'IAmAStringThreadAWasHere'. I am assuming this should not be the case.
> We should have 'ThreadBWasHereThreadAWasHere' in the final string,
> because we have just consumed the code in ThreadB.
> 
Assuming that thread A and B contain nothing but that code, you will
either get 'ThreadBWasHereThreadAWasHere' or just 'ThreadBWasHere',
because by default the interpreter switches threads every 100 bytecode
instructions.
> Also, the other question is the operation st = 'ThreadBWasHere' is
> atomic? I mean, if Python does not guarantee if an immutable object
> assignment is atomic, then how can we say that the object is thread-
> safe? So, if it is an atomic operation, which operators are atomic,
> means only assignment'='? How about other operators like +=, or -
> =..etc?
The augmented assignment operators first load the current value of the
variable like this:
a += 'asdf'
becomes
  1   0 LOAD_NAME0 (a)
  3 LOAD_CONST   0 ('asdf')
  6 INPLACE_ADD
  7 STORE_NAME   0 (a)
> 
> I am confused about which data structure to rely on thread-safety, or
> operator in Python?
All of the builtin functions(which are implemented in C, like len()) are
atomic(but assigning their output to a value may not be).

I hope this helps,
AlcariTheMad
--
http://mail.python.org/mailman/listinfo/python-list

Re: Immutable object thread-safety

[Alcari The Mad]

Laszlo Nagy wrote:
> 
>> Also, the other question is the operation st = 'ThreadBWasHere' is
>> atomic? 
> I think this is the same question. And I believe it is not atomic,
> because it is actually rebinding a name. Consider this:
> 
> a,b = b,a
> 
> This will rebind both a and b. In order to be correct, it MUST happen in
> two phases: first calculate the right side, then do the rebind to the
> names on the left side. The expression on the right side can be anything
> that executes for a long time, and can even rebind 'a' and 'b' several
> times, and will probably be paused by other threads etc. So the
> assignment above cannot be atomic.

You are correct.
a,b = b,a
yields the bytecode:

  1   0 LOAD_NAME0 (b)
  3 LOAD_NAME1 (a)
  6 ROT_TWO
  7 STORE_NAME   1 (a)
 10 STORE_NAME   0 (b)

Which is most definitely not atomic.

> 
> I strongly feel that if such an assignment is not atomic for "a,b = b,a"
> then int wont be atomic for "a+=b" or eveb "a=b" or any other
> assignment. However, I can be completely wrong. :-)

'a = b' requires two bytecodes:

  1   0 LOAD_NAME0 (b)
  3 STORE_NAME   1 (a)

>> I mean, if Python does not guarantee if an immutable object
>> assignment is atomic, then how can we say that the object is thread-
>> safe? 
> An assigment rebinds name to an object. It is assignment to a name.
> Assigning immutable and mutable objects to names are not specially
> distinguished.
>> So, if it is an atomic operation, which operators are atomic,
>> means only assignment'='? 
> I don't think that '=' operator is atomic, see above.
>> I am confused about which data structure to rely on thread-safety, or
>> operator in Python?
>>   
> The immutable object itself will never change state, so it is thread
> safe. The problem is not with the object, but the dictionary which holds
> the name 'b' and 'a' and 'st' in your example. It is mutable, and so
> must be protected in a threaded application.

I would suggest reading up on using the 'with' statement and
thread.allocate_lock().

> 
> Best,
> 
>  Laszlo
> 
--
http://mail.python.org/mailman/listinfo/python-list