The link to the repo does not allow not credentialed access... > On Apr 7, 2019, at 5:58 AM, Tharen Abela <abela.tha...@gmail.com> wrote: > > I'll preface this by saying this is for academic purposes. > > I am working in the following repository > <https://gitlab.com/AbelThar/go.batch>, but the following files are the ones > relevant, I'll reference them as I explain what I have: > > 0. go 1.11.5 commit 7cf31d8f `proc.go` > <https://gitlab.com/AbelThar/go.batch/blob/7cf31d8f4116420e396c5e8690c043b2ce83f90a/src/runtime/proc.go> > `runtime2.go` > <https://gitlab.com/AbelThar/go.batch/blob/7cf31d8f4116420e396c5e8690c043b2ce83f90a/src/runtime/runtime2.go> > 1. Modified with 1 goroutine `proc.go` > <https://gitlab.com/AbelThar/go.batch/blob/b10ef431c29b01fa7568a7bf9712a0286033266f/src/runtime/proc.go> > `runtime2.go` > <https://gitlab.com/AbelThar/go.batch/blob/b10ef431c29b01fa7568a7bf9712a0286033266f/src/runtime/runtime2.go> > 2. Modified with 1 goroutine per batch `proc.go` > <https://gitlab.com/AbelThar/go.batch/blob/c066d64315f1da3bb20976456929fec00673d70f/src/runtime/proc.go> > `runtime2.go` > <https://gitlab.com/AbelThar/go.batch/blob/c066d64315f1da3bb20976456929fec00673d70f/src/runtime/runtime2.go> > > > For convenience, here are the diffs: 0 <-> 1 > <https://www.diffchecker.com/qfIN3GCW>, 1 <-> 2 > <https://www.diffchecker.com/xNFmlNCl> > I have provided some quick sketches to explain further. > > The first is a representation of the current scheduler, with Local Runqueues > and the P locks for stealing (0). > <gostd.png> > > > > > > From this point I modified it by (files at 1): > - Removing Work-Stealing > - Modified the Local Run Queue to only keep one goroutine > > (look at 0 <-> 1 diff for more detail) > > This works as expected, no benchmarks fail etc... (benchmarks using the ones > provided from shootout - binarytree, fannkuch, etc.) > Sketch for reference: > > <oneG.png> > > In 2, I want to have a batch struct, in which the goroutine is contained, > with the global run queue replaced by a global batch queue, for now I am > keeping it at a 1 goroutine, but the point is to *batch* a number of > goroutines together. > > So for now it will simply serve as overhead. > > <batch.png> > > > bQueue is the same as gQueue, just adapted for batches. The GRQ has been > changed to a Global Batch Queue, with type bQueue, and respective operations > make use of the batches; globrunqput first puts the G in a batch, then in the > GBQ, and globrunqget gets a batch, and pops the G within it. (look at 1 <-> 2 > diff for more detail) > > When it came to the batch solution, I have a bunch of extra batches > pre-allocated, which a goroutine is placed in, when put in the global queue. > This is mainly due that a P is required to allocate a batch, and at > exitsyscall, it cannot acquire a P. Its also an optimization, as it will > reduce the number of batch objects created. > > The current problem I am encountering is that the GC occasionally incorrectly > sweeps the batch object. To allocate I do use the `acquirem`, similarly used > in acquireSudog. I also keep an `allb` slice, with pointers to all batches, > similar to `allp` and `allm`. > > By occasionally, I mean that about 1/3 times it does work, and for programs > that do not invoke the GC, it consistently works. > > When it does crash, running it by default, I get: > runtime: nelems=512 nalloc=2 previous allocCount=1 nfreed=65535 > fatal error: sweep increased allocation count > The nalloc and allocCount is always x and x+1, respectively (not only 2 and > 1). > > Running with `GODEBUG=gccheckmark=1`, I get the following output: > runtime: marking free object 0xc000448020 found at *(0xc0004d8000+0x48) > base=0xc0004d8000 s.base()=0xc0004d8000 s.limit=0xc0004da000 s.spanclass=18 > s.elemsize=128 s.state=mSpanInUse > *(base+0) = 0xc0000160f0 > *(base+8) = 0xc000016100 > *(base+16) = 0xc000016110 > *(base+24) = 0xc000016120 > *(base+32) = 0xc000068000 > *(base+40) = 0xc000068010 > *(base+48) = 0xc00046c000 > *(base+56) = 0xc000068060 > *(base+64) = 0xc00046c010 > *(base+72) = 0xc000448020 <== > *(base+80) = 0x0 > *(base+88) = 0x0 > *(base+96) = 0x0 > *(base+104) = 0x0 > *(base+112) = 0x0 > *(base+120) = 0x0 > obj=0xc000448020 s.base()=0xc000448000 s.limit=0xc00044a000 s.spanclass=5 > s.elemsize=16 s.state=mSpanInUse > *(obj+0) = 0x0 > *(obj+8) = 0xc00046c010 > fatal error: marking free object > > > The pointer of the object is a batch type (checked with printing the pointer > when allocated) > > I'm not sure if there is anything I missed in the allocation process for > safety; I don't mind not so ideal solution like an inefficient lock or > something along those lines. > I appreciate any help I can get. > > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com > <mailto:golang-nuts+unsubscr...@googlegroups.com>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. > <gostd.png><oneG.png><batch.png>
-- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
