I’m sorry, but it’s very hard to understand when you start with solutions. I think maybe clearly restating the problem will allow more people to offer up ideas. To be honest at this point I’m not really certain what you’re trying to demonstrate or why.
> On Dec 8, 2019, at 12:44 AM, Egon Kocjan <ekoc...@gmail.com> wrote: > > I meant lock-free as in "without explicit locks". > > The original challenge still stands if someone has a better solution than me: > "The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong > implementation of bidi-comm, which is what I'll be illustrating. I have three > working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can we remove > the extra goroutine from 1_1.go and make the code nicer to read than 2_3.go > and 2_4.go. The extra goroutine that I'd like to be removed is started here: > https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14)" > >> On Sunday, December 8, 2019 at 7:18:16 AM UTC+1, Robert Engels wrote: >> I understand what you are saying but I’ll still suggest that your >> premise/design is not correct. There are plenty of useful lock free >> structures in Go (see github.com/robaho/go-concurrency-test) but that is not >> what you are attempting here... you are using async processing - these are >> completely different things. Using async in Go is an anti-pattern IMO. >> >>> On Dec 8, 2019, at 12:11 AM, Egon Kocjan <eko...@gmail.com> wrote: >>> >>> >>> I'll cite myself: >>> "I'm preparing a short talk about Go channels and select. More >>> specifically, I want to show what not to do." >>> and >>> "it would be tempting to just combine two goroutines into one and handle >>> caching in a single loop without using locks (I see developers avoid >>> atomics and locks if they don't have a lot of previous experience with >>> traditional MT primitives)" >>> >>> Before I say one can't do something in Go, I wanted to ask here to make >>> sure I'm not missing something obvious. Basically, I intend to show how >>> difficult lock-free programming can be so don't force it - just use >>> goroutines and locks. >>> >>>> On Saturday, December 7, 2019 at 3:46:43 PM UTC+1, Robert Engels wrote: >>>> Probably not. Go is designed for 1:1 and there is no reason to do it >>>> differently. You could probably try to write an async event driven layer >>>> (which it looks like you’ve tried) but why??? >>>> >>>> It’s like saying I’d really like my plane to float - you can do that -but >>>> most likely you want a boat instead of a plane. >>>> >>>>> On Dec 7, 2019, at 2:38 AM, Egon Kocjan <eko...@gmail.com> wrote: >>>>> >>>>> >>>>> I'll try to clarify as best as I can, thanks again to anyone looking at >>>>> this. >>>>> >>>>> The simple server implementation of "output <- input+1" is here and it is >>>>> not "under our control" - it's what we have to work with: >>>>> https://github.com/egonk/chandemo/blob/master/server.go >>>>> >>>>> The test runner or client is here: >>>>> https://github.com/egonk/chandemo/blob/master/demo.go (it just pushes in >>>>> ints and gets server replies back through a connection layer) >>>>> >>>>> The deadlocks in 2_1.go and 2_2.go are caused by the simplistic and wrong >>>>> implementation of bidi-comm, which is what I'll be illustrating. I have >>>>> three working solutions - 1_1.go, 2_3.go, 2_4.go. So the question is, can >>>>> we remove the extra goroutine from 1_1.go and make the code nicer to read >>>>> than 2_3.go and 2_4.go. The extra goroutine that I'd like to be removed >>>>> is started here: >>>>> https://github.com/egonk/chandemo/blob/master/1_1.go#L14 (line 14) >>>>> >>>>> What I mean by removed - no go statement, replaced presumably by some >>>>> kind of for/select combination. >>>>> >>>>>> On Saturday, December 7, 2019 at 7:02:50 AM UTC+1, robert engels wrote: >>>>>> I’m sorry but your design is not comprehendible by me, and I’ve done >>>>>> lots of TCP based services. >>>>>> >>>>>> i think you only need to emulate classic TCP processing - a reader >>>>>> thread (Go routine) on each side of the connection using range to read >>>>>> until closed. The connection is represented by 2 channels - one for each >>>>>> direction. >>>>>> >>>>>> I think you might be encountering a deadlock because the producer on one >>>>>> end is not also reading the incoming - so either restructure, or use 2 >>>>>> more threads for the producers. >>>>>> >>>>>> >>>>>> >>>>>>> On Dec 6, 2019, at 10:38 PM, Egon Kocjan <eko...@gmail.com> wrote: >>>>>>> >>>>>>> Agreed, I see goroutines in general as a big win. But what I intend to >>>>>>> talk about in the presentation: >>>>>>> - we have two unidirectional flows of data resembling something like a >>>>>>> TCP socket, easy to do with two goroutines with a for loop >>>>>>> - let's add caching, so some requests do not go to the server >>>>>>> - it would be tempting to just combine two goroutines into one and >>>>>>> handle caching in a single loop without using locks (I see developers >>>>>>> avoid atomics and locks if they don't have a lot of previous experience >>>>>>> with traditional MT primitives) >>>>>>> - this is surprisingly difficult to do properly with Go channels, see >>>>>>> my attempts: https://github.com/egonk/chandemo/blob/master/2_3.go and >>>>>>> https://github.com/egonk/chandemo/blob/master/2_4.go >>>>>>> - it is easy to do in actor systems, just move the code for both actors >>>>>>> into a single actor! >>>>>>> >>>>>>> The lesson here is that select is not a nice and safe compose statement >>>>>>> even if it appears so at the first glance, do not be afraid to use >>>>>>> locks. >>>>>>> >>>>>>> Of course, if somebody comes up with a better implementation than >>>>>>> 2_3.go and 2_4.go, I would be very happy to include it in the talk. >>>>>>> >>>>>>>> On Saturday, December 7, 2019 at 4:17:04 AM UTC+1, robert engels wrote: >>>>>>>> To clarify, with Go’s very lightweight threads it is “doing the >>>>>>>> multiplexing for you” - often only a single CPU is consumed if the >>>>>>>> producer and consumer work cannot be parallelized, otherwise you get >>>>>>>> this concurrency “for free”. >>>>>>>> >>>>>>>> You are trying to manually perform the multiplexing - you need async >>>>>>>> structures to do this well - Go doesn’t really support async by design >>>>>>>> - and it’s a much simpler programming model as a result. >>>>>>>> >>>>>>>>> On Dec 6, 2019, at 12:02 PM, Robert Engels <ren...@ix.netcom.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> A channel is much closer to a pipe. There are producers and consumers >>>>>>>>> and these are typically different threads of execution unless you >>>>>>>>> have an event based (async) system - that is not Go. >>>>>>>>> >>>>>>>>>> On Dec 6, 2019, at 9:30 AM, Egon Kocjan <eko...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> There are goroutines in the examples of course, just a single >>>>>>>>>> goroutine per bidi channel seems hard. By contrast, I've worked with >>>>>>>>>> actor systems before and they are perfectly fine with a single fiber. >>>>>>>>>> >>>>>>>>>>> On Friday, December 6, 2019 at 3:38:20 PM UTC+1, Robert Engels >>>>>>>>>>> wrote: >>>>>>>>>>> Channels are designed to be used with multiple go routines - if >>>>>>>>>>> you’re not you are doing something wrong. >>>>>>>>>>> >>>>>>>>>>>> On Dec 6, 2019, at 8:32 AM, Egon Kocjan <eko...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Hello >>>>>>>>>>>> >>>>>>>>>>>> I'm preparing a short talk about Go channels and select. More >>>>>>>>>>>> specifically, I want to show what not to do. I chose a >>>>>>>>>>>> bidirectional communication channel implementation, because it >>>>>>>>>>>> seems to be a common base for a lot of problems but hard to >>>>>>>>>>>> implement correctly without using any extra goroutines. All the >>>>>>>>>>>> code is here: https://github.com/egonk/chandemo >>>>>>>>>>>> >>>>>>>>>>>> 1_1.go: easy with en extra goroutine (takes 1.2s for million ints) >>>>>>>>>>>> 2_1.go: nice but completely wrong >>>>>>>>>>>> 2_2.go: better but still deadlocks >>>>>>>>>>>> 2_3.go: correct but ugly and slow (takes more than 2s for million >>>>>>>>>>>> ints) >>>>>>>>>>>> 2_4.go: correct and a bit faster but still ugly (1.8s for million >>>>>>>>>>>> ints) >>>>>>>>>>>> >>>>>>>>>>>> So my question: is there a better way of doing it with just nested >>>>>>>>>>>> for and select and no goroutines? Basically, what would 2_5.go >>>>>>>>>>>> look like? >>>>>>>>>>>> >>>>>>>>>>>> Thank you >>>>>>>>>>>> Egon >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>>>> Groups "golang-nuts" group. >>>>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>>>> send an email to golan...@googlegroups.com. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/82830a5d-2bd8-4324-890e-9ae7f5f0fbaf%40googlegroups.com. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the Google >>>>>>>>>> Groups "golang-nuts" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to golan...@googlegroups.com. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/msgid/golang-nuts/bdc57eb0-b26f-4364-87fb-241b0807e8ae%40googlegroups.com. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "golang-nuts" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>>>> an email to golan...@googlegroups.com. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/golang-nuts/75d69b4e-4fb7-4f62-8011-f21e2a4c294a%40googlegroups.com. >>>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google Groups >>>>> "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send an >>>>> email to golan...@googlegroups.com. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/golang-nuts/8b87adcc-2249-402c-b34c-20df5013860a%40googlegroups.com. >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to golan...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/golang-nuts/3b9bb722-d43f-4e70-8384-dc17cdec6090%40googlegroups.com. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/golang-nuts/4a176af0-74bb-49b5-ae4d-d8714c7bc46d%40googlegroups.com. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/694D5BCB-F3A0-4918-8DDD-484961892B4E%40ix.netcom.com.
