On Saturday, November 5, 2016 at 7:48:27 AM UTC-7, leob...@gmail.com wrote:
>
> Hey,
>
> I have a scenario where I need to iterate over (as many as possible) map
> entries and send them into a channel.
> The operation on the other end of the channel can take a long time, so
> sends on the channel may block for a long time. The map is protected by an
> RWMutex.
> The map is also rather big and I want to avoid creating a temporary copy
> of it.
>
> Assume I have a struct like this:
> type Example struct {
> sync.RWMutex
> m map[string]struct{}
> }
>
> Now I came up with something like this:
>
> func (e *Example) StreamAll() <-chan string {
> toReturn := make(chan string)
> go func() {
> e.RLock()
> defer e.RUnlock()
> for k := range e.m {
> e.RUnlock()
> toReturn <- k
> e.RLock()
> }
> close(toReturn)
> }()
> return toReturn
> }
>
>
>
> The language specification <https://golang.org/ref/spec#For_statements> has
> this interesting bit about ranging over maps:
>
>> If map entries that have not yet been reached are removed during
>> iteration, the corresponding iteration values will not be produced. If map
>> entries are created during iteration, that entry may be produced during the
>> iteration or may be skipped.
>>
>
> Now, I was wondering:
> 1. Does my method of concurrently streaming the map entries cause any
> problems? (Especially if the map is changed between iterations of the for
> loop)
>
You cannot modify the map in another goroutine while iterating. Iterating
is considered a read operation for the entire duration of the read. Writes
happening simultaneously with reads are a data race.
"Entire duration" is the thing that's tripping you up, I think. You can't
release the read lock inside the for loop.
I'm afraid that for your use case you would need to read all the entries
(at least the keys) out of the map under a single read lock.
Note that modifying the map in the same goroutine while iterating is ok.
> 2. What happens if the key-value pair I just pulled out of the map is
> deleted while blocking on the channel send?
>
This is ok. Once you read a key/value out of the map, they are yours.
They are copies of the map entries. The fact that they came from a map is
irrelevant.
> 3. Is there anything known about the amount of key-value pairs I reach if
> the map is modified between iterations?
>
Only what you quoted above. If you keep inserting new entries iteration
may go on forever, for example.
Here is a complete example on the playground
> <https://play.golang.org/p/aktX2Nta5s>.
>
> Best,
> Leo
>
>
>
>
>
>
>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
