I think it would be helpful to if you providee a summary of the issue/idea before the detailed information - it makes it nearly impossible to understand - at least for me.
> On Apr 7, 2019, at 1:41 PM, robert engels <reng...@ix.netcom.com> wrote: > > The link to the repo does not allow not credentialed access... > >> On Apr 7, 2019, at 5:58 AM, Tharen Abela <abela.tha...@gmail.com >> <mailto:abela.tha...@gmail.com>> wrote: >> >> I'll preface this by saying this is for academic purposes. >> >> I am working in the following repository >> <https://gitlab.com/AbelThar/go.batch>, but the following files are the ones >> relevant, I'll reference them as I explain what I have: >> >> 0. go 1.11.5 commit 7cf31d8f `proc.go` >> <https://gitlab.com/AbelThar/go.batch/blob/7cf31d8f4116420e396c5e8690c043b2ce83f90a/src/runtime/proc.go> >> `runtime2.go` >> <https://gitlab.com/AbelThar/go.batch/blob/7cf31d8f4116420e396c5e8690c043b2ce83f90a/src/runtime/runtime2.go> >> 1. Modified with 1 goroutine `proc.go` >> <https://gitlab.com/AbelThar/go.batch/blob/b10ef431c29b01fa7568a7bf9712a0286033266f/src/runtime/proc.go> >> `runtime2.go` >> <https://gitlab.com/AbelThar/go.batch/blob/b10ef431c29b01fa7568a7bf9712a0286033266f/src/runtime/runtime2.go> >> 2. Modified with 1 goroutine per batch `proc.go` >> <https://gitlab.com/AbelThar/go.batch/blob/c066d64315f1da3bb20976456929fec00673d70f/src/runtime/proc.go> >> `runtime2.go` >> <https://gitlab.com/AbelThar/go.batch/blob/c066d64315f1da3bb20976456929fec00673d70f/src/runtime/runtime2.go> >> >> >> For convenience, here are the diffs: 0 <-> 1 >> <https://www.diffchecker.com/qfIN3GCW>, 1 <-> 2 >> <https://www.diffchecker.com/xNFmlNCl> >> I have provided some quick sketches to explain further. >> >> The first is a representation of the current scheduler, with Local Runqueues >> and the P locks for stealing (0). >> <gostd.png> >> >> >> >> >> >> From this point I modified it by (files at 1): >> - Removing Work-Stealing >> - Modified the Local Run Queue to only keep one goroutine >> >> (look at 0 <-> 1 diff for more detail) >> >> This works as expected, no benchmarks fail etc... (benchmarks using the ones >> provided from shootout - binarytree, fannkuch, etc.) >> Sketch for reference: >> >> <oneG.png> >> >> In 2, I want to have a batch struct, in which the goroutine is contained, >> with the global run queue replaced by a global batch queue, for now I am >> keeping it at a 1 goroutine, but the point is to *batch* a number of >> goroutines together. >> >> So for now it will simply serve as overhead. >> >> <batch.png> >> >> >> bQueue is the same as gQueue, just adapted for batches. The GRQ has been >> changed to a Global Batch Queue, with type bQueue, and respective operations >> make use of the batches; globrunqput first puts the G in a batch, then in >> the GBQ, and globrunqget gets a batch, and pops the G within it. (look at 1 >> <-> 2 diff for more detail) >> >> When it came to the batch solution, I have a bunch of extra batches >> pre-allocated, which a goroutine is placed in, when put in the global queue. >> This is mainly due that a P is required to allocate a batch, and at >> exitsyscall, it cannot acquire a P. Its also an optimization, as it will >> reduce the number of batch objects created. >> >> The current problem I am encountering is that the GC occasionally >> incorrectly sweeps the batch object. To allocate I do use the `acquirem`, >> similarly used in acquireSudog. I also keep an `allb` slice, with pointers >> to all batches, similar to `allp` and `allm`. >> >> By occasionally, I mean that about 1/3 times it does work, and for programs >> that do not invoke the GC, it consistently works. >> >> When it does crash, running it by default, I get: >> runtime: nelems=512 nalloc=2 previous allocCount=1 nfreed=65535 >> fatal error: sweep increased allocation count >> The nalloc and allocCount is always x and x+1, respectively (not only 2 and >> 1). >> >> Running with `GODEBUG=gccheckmark=1`, I get the following output: >> runtime: marking free object 0xc000448020 found at *(0xc0004d8000+0x48) >> base=0xc0004d8000 s.base()=0xc0004d8000 s.limit=0xc0004da000 s.spanclass=18 >> s.elemsize=128 s.state=mSpanInUse >> *(base+0) = 0xc0000160f0 >> *(base+8) = 0xc000016100 >> *(base+16) = 0xc000016110 >> *(base+24) = 0xc000016120 >> *(base+32) = 0xc000068000 >> *(base+40) = 0xc000068010 >> *(base+48) = 0xc00046c000 >> *(base+56) = 0xc000068060 >> *(base+64) = 0xc00046c010 >> *(base+72) = 0xc000448020 <== >> *(base+80) = 0x0 >> *(base+88) = 0x0 >> *(base+96) = 0x0 >> *(base+104) = 0x0 >> *(base+112) = 0x0 >> *(base+120) = 0x0 >> obj=0xc000448020 s.base()=0xc000448000 s.limit=0xc00044a000 s.spanclass=5 >> s.elemsize=16 s.state=mSpanInUse >> *(obj+0) = 0x0 >> *(obj+8) = 0xc00046c010 >> fatal error: marking free object >> >> >> The pointer of the object is a batch type (checked with printing the pointer >> when allocated) >> >> I'm not sure if there is anything I missed in the allocation process for >> safety; I don't mind not so ideal solution like an inefficient lock or >> something along those lines. >> I appreciate any help I can get. >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts+unsubscr...@googlegroups.com >> <mailto:golang-nuts+unsubscr...@googlegroups.com>. >> For more options, visit https://groups.google.com/d/optout >> <https://groups.google.com/d/optout>. >> <gostd.png><oneG.png><batch.png> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
