On Sunday, 6 November 2016 05:00:33 UTC+1, Keith Randall wrote:
>
>
>
> On Saturday, November 5, 2016 at 7:48:27 AM UTC-7, leob...@gmail.com
> wrote:
>>
>> Hey,
>>
>> I have a scenario where I need to iterate over (as many as possible) map
>> entries and send them into a channel.
>> The operation on the other end of the channel can take a long time, so
>> sends on the channel may block for a long time. The map is protected by an
>> RWMutex.
>> The map is also rather big and I want to avoid creating a temporary copy
>> of it.
>>
>> Assume I have a struct like this:
>> type Example struct {
>> sync.RWMutex
>> m map[string]struct{}
>> }
>>
>> Now I came up with something like this:
>>
>> func (e *Example) StreamAll() <-chan string {
>> toReturn := make(chan string)
>> go func() {
>> e.RLock()
>> defer e.RUnlock()
>> for k := range e.m {
>> e.RUnlock()
>> toReturn <- k
>> e.RLock()
>> }
>> close(toReturn)
>> }()
>> return toReturn
>> }
>>
>>
>>
>> The language specification <https://golang.org/ref/spec#For_statements> has
>> this interesting bit about ranging over maps:
>>
>>> If map entries that have not yet been reached are removed during
>>> iteration, the corresponding iteration values will not be produced. If map
>>> entries are created during iteration, that entry may be produced during the
>>> iteration or may be skipped.
>>>
>>
>> Now, I was wondering:
>> 1. Does my method of concurrently streaming the map entries cause any
>> problems? (Especially if the map is changed between iterations of the for
>> loop)
>>
>
> You cannot modify the map in another goroutine while iterating. Iterating
> is considered a read operation for the entire duration of the read. Writes
> happening simultaneously with reads are a data race.
> "Entire duration" is the thing that's tripping you up, I think. You can't
> release the read lock inside the for loop.
> I'm afraid that for your use case you would need to read all the entries
> (at least the keys) out of the map under a single read lock.
>
> Note that modifying the map in the same goroutine while iterating is ok.
>
>
>> 2. What happens if the key-value pair I just pulled out of the map is
>> deleted while blocking on the channel send?
>>
>
> This is ok. Once you read a key/value out of the map, they are yours.
> They are copies of the map entries. The fact that they came from a map is
> irrelevant.
>
>
>> 3. Is there anything known about the amount of key-value pairs I reach if
>> the map is modified between iterations?
>>
>
> Only what you quoted above. If you keep inserting new entries iteration
> may go on forever, for example.
>
> Here is a complete example on the playground
>> <https://play.golang.org/p/aktX2Nta5s>.
>>
>> Best,
>> Leo
>>
>>
>>
>>
>>
>>
>>
>>
Hey,
thank you for clearing that up!
I was thinking about how for range loops work and apparently got a little
ahead of myself there :)
Cheers,
Leo
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
