Re: Thread safety of C++ Struct/Schema::GetFieldIndex

2019-01-04 Thread Gene Novark
on't feel put upon if you fix it before then. Thanks for the feedback/input! -Original Message- From: Antoine Pitrou Sent: Friday, January 4, 2019 6:00 PM To: dev@arrow.apache.org Subject: (EXT) Re: Thread safety of C++ Struct/Schema::GetFieldIndex However, eager initialization c

Re: Thread safety of C++ Struct/Schema::GetFieldIndex

2019-01-04 Thread Antoine Pitrou
However, eager initialization could probably work too. I'm not sure why we used lazy initialization like this. Perhaps someone worried about the cost of incremental schema construction using repeated Schema::AddField() calls (but that's gonna be wasteful anyway)? Le 04/01/2019 à 23:42, Antoi

Re: Thread safety of C++ Struct/Schema::GetFieldIndex

2019-01-04 Thread Antoine Pitrou
Generally speaking, normal assignments are not thread-safe. Intuitively they could be (and perhaps in some simple cases - such as aligned machine types - they will turn up to be on some specific compiler/CPU combinations), but C++ makes no guarantee about that (for example, an assignment can be

Re: Thread safety of C++ Struct/Schema::GetFieldIndex

2019-01-04 Thread Antoine Pitrou
In other words, something like: class StructType { mutable std::shared_ptr> name_to_index_; std::shared_ptr> GetNameToIndex() const { if (!std:atomic_load(&name_to_index_)) { name_to_index = std::make_shared>(); // ... initialize name_to_index ... std::atomic_store(&na

Re: Thread safety of C++ Struct/Schema::GetFieldIndex

2019-01-04 Thread Wes McKinney
That works. I would have thought that the deterministic state of unordered_map might make move-assignment safe, but perhaps not On Fri, Jan 4, 2019 at 4:33 PM Antoine Pitrou wrote: > > > The move-assigning would definitely not be thread-safe. > > One possibility would be to wrap the std::unordere

Re: Thread safety of C++ Struct/Schema::GetFieldIndex

2019-01-04 Thread Antoine Pitrou
The move-assigning would definitely not be thread-safe. One possibility would be to wrap the std::unordered_map in a std::shared_ptr, and use the atomic functions for shared_ptr: https://en.cppreference.com/w/cpp/memory/shared_ptr/atomic Regards Antoine. Le 04/01/2019 à 23:17, Wes McKinney a

Re: Thread safety of C++ Struct/Schema::GetFieldIndex

2019-01-04 Thread Wes McKinney
hi Gene, Yes, feel free to submit a PR to fix this. I would suggest populating function-local std::unordered_map and then move-assigning it into name_to_index_ -- I think this should not have race conditions. If you do want to add a mutex, it could be a static one rather than creating a new mutex

Thread safety of C++ Struct/Schema::GetFieldIndex

2019-01-04 Thread Gene Novark
These are both effectively-immutable accessors with lazy initialization. However, when accessed from multiple threads a race can occur initializing the name_to_index_ map. This seems like a bug rather than a purposeful design choice based off the cpp/conventions.rst section on Immutability. I'm