Should be public now, thanks for bringing that to my attention. On Sunday, 7 April 2019 20:41:41 UTC+2, robert engels wrote: > > The link to the repo does not allow not credentialed access... > > On Apr 7, 2019, at 5:58 AM, Tharen Abela <abela...@gmail.com <javascript:>> > wrote: > > I'll preface this by saying this is for academic purposes. > > I am working in the following repository > <https://gitlab.com/AbelThar/go.batch>, but the following files are the > ones relevant, I'll reference them as I explain what I have: > > *0*. go 1.11.5 commit 7cf31d8f `proc.go` > <https://gitlab.com/AbelThar/go.batch/blob/7cf31d8f4116420e396c5e8690c043b2ce83f90a/src/runtime/proc.go> > > `runtime2.go` > <https://gitlab.com/AbelThar/go.batch/blob/7cf31d8f4116420e396c5e8690c043b2ce83f90a/src/runtime/runtime2.go> > *1*. Modified with 1 goroutine `proc.go` > <https://gitlab.com/AbelThar/go.batch/blob/b10ef431c29b01fa7568a7bf9712a0286033266f/src/runtime/proc.go> > > `runtime2.go` > <https://gitlab.com/AbelThar/go.batch/blob/b10ef431c29b01fa7568a7bf9712a0286033266f/src/runtime/runtime2.go> > *2*. Modified with 1 goroutine per batch `proc.go` > <https://gitlab.com/AbelThar/go.batch/blob/c066d64315f1da3bb20976456929fec00673d70f/src/runtime/proc.go> > > `runtime2.go` > <https://gitlab.com/AbelThar/go.batch/blob/c066d64315f1da3bb20976456929fec00673d70f/src/runtime/runtime2.go> > > > For convenience, here are the diffs: 0 <-> 1 > <https://www.diffchecker.com/qfIN3GCW>, 1 <-> 2 > <https://www.diffchecker.com/xNFmlNCl> > > I have provided some quick sketches to explain further. > > The first is a representation of the current scheduler, with Local > Runqueues and the P locks for stealing (*0*). > > <gostd.png> > > > > > From this point I modified it by (files at *1*): > - Removing Work-Stealing > - Modified the Local Run Queue to only keep one goroutine > > (look at 0 <-> 1 diff for more detail) > > This works as expected, no benchmarks fail etc... (benchmarks using the > ones provided from shootout - binarytree, fannkuch, etc.) > Sketch for reference: > > <oneG.png> > In *2*, I want to have a batch struct, in which the goroutine is > contained, with the global run queue replaced by a global batch queue, for > now I am keeping it at a 1 goroutine, but the point is to *batch* a number > of goroutines together. > > So for now it will simply serve as overhead. > > <batch.png> > > bQueue is the same as gQueue, just adapted for batches. The GRQ has been > changed to a Global Batch Queue, with type bQueue, and respective > operations make use of the batches; globrunqput first puts the G in a > batch, then in the GBQ, and globrunqget gets a batch, and pops the G within > it. (look at 1 <-> 2 diff for more detail) > > When it came to the batch solution, I have a bunch of extra batches > pre-allocated, which a goroutine is placed in, when put in the global > queue. This is mainly due that a P is required to allocate a batch, and at > exitsyscall, it cannot acquire a P. Its also an optimization, as it will > reduce the number of batch objects created. > > The current problem I am encountering is that the GC occasionally > incorrectly sweeps the batch object. To allocate I do use the `acquirem`, > similarly used in acquireSudog. I also keep an `allb` slice, with pointers > to all batches, similar to `allp` and `allm`. > > By occasionally, I mean that about 1/3 times it does work, and for > programs that do not invoke the GC, it consistently works. > > When it does crash, running it by default, I get: > runtime: nelems=512 nalloc=2 previous allocCount=1 nfreed=65535 > fatal error: sweep increased allocation count > The nalloc and allocCount is always x and x+1, respectively (not only 2 > and 1). > > Running with `GODEBUG=gccheckmark=1`, I get the following output: > runtime: marking free object 0xc000448020 found at *(0xc0004d8000+0x48) > base=0xc0004d8000 s.base()=0xc0004d8000 s.limit=0xc0004da000 s.spanclass= > 18 s.elemsize=128 s.state=mSpanInUse > *(base+0) = 0xc0000160f0 > *(base+8) = 0xc000016100 > *(base+16) = 0xc000016110 > *(base+24) = 0xc000016120 > *(base+32) = 0xc000068000 > *(base+40) = 0xc000068010 > *(base+48) = 0xc00046c000 > *(base+56) = 0xc000068060 > *(base+64) = 0xc00046c010 > *(base+72) = 0xc000448020 <== > *(base+80) = 0x0 > *(base+88) = 0x0 > *(base+96) = 0x0 > *(base+104) = 0x0 > *(base+112) = 0x0 > *(base+120) = 0x0 > obj=0xc000448020 s.base()=0xc000448000 s.limit=0xc00044a000 s.spanclass=5 > s.elemsize=16 s.state=mSpanInUse > *(obj+0) = 0x0 > *(obj+8) = 0xc00046c010 > fatal error: marking free object > > > The pointer of the object is a batch type (checked with printing the > pointer when allocated) > > I'm not sure if there is anything I missed in the allocation process for > safety; I don't mind not so ideal solution like an inefficient lock or > something along those lines. > I appreciate any help I can get. > > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golan...@googlegroups.com <javascript:>. > For more options, visit https://groups.google.com/d/optout. > <gostd.png><oneG.png><batch.png> > > >
-- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
