Doug, you're welcome! Yeah, my simple "two buffers back to back" example is a bit on the simplistic side. Actually, we have multiple implementations on how to get mem-mappable pages, and how much copies/"abort if someone accidentially ends up in this memory region" pages we have, and of course Windows is a bit special, but in principle: Here's your magic gr_buffer, demagified.
By the way: This currently *is* getting more interesting: Because you typically don't want to copy memory needlessly in a performance-critical application, it's bad that blocks that wrap some kind of accelerator (GPU, FPGA card, DSP core...) can't define where their buffers are -- so there's work going on in the coprocessors working group (Doug Geiger is the person to ask, I guess) to allow single blocks to define their own special buffers. Greetings, Marcus On 04/21/2015 08:50 PM, Anderson, Douglas J. wrote: > Marcus, > > Awesome! I'm working my way through the scheduler slides linked > earlier today, and I get to where it's describing buffers on linux > like "attach shmid1 to first half of shmid2 with shmat, attach shmid1 > to second half of shmid2 with shmat, memory in both halves of shmid2 > are mapped to the same virtual space..." and I was just really lost. I > appreciate the diagram; now it's making sense that the circular buffer > is implemented with 2 copies of "shmid1" back to back. I see now that > the page size if just a requirement of the tools used to create this > clever buffer: > > *shmget*() returns the identifier of the shared memory segment > associated with the value of the argument /key/. A new shared memory > segment, with size equal to the value of /size/ rounded up to a > multiple of *PAGE_SIZE* > * > * > Very cool stuff. Thanks! > -Doug > ------------------------------------------------------------------------ > *From:* discuss-gnuradio-bounces+danderson=its.bldrdoc....@gnu.org > [discuss-gnuradio-bounces+danderson=its.bldrdoc....@gnu.org] on behalf > of Marcus Müller [marcus.muel...@ettus.com] > *Sent:* Tuesday, April 21, 2015 12:33 PM > *To:* discuss-gnuradio@gnu.org > *Subject:* Re: [Discuss-gnuradio] gr::buffer::allocate_buffer: warning > > Hi Doug, > > ok, you asked for this :D > So, GNU Radio's buffers look a lot like real circular buffers to the > blocks using them: > For example, assume your buffer between source block A and sink block > B is large enough to store exactly 10000 of your items: > A-->B > Now, A has produced 9000 items, of which B has already consumed 8000. > Your buffer thus looks like this: > > - - - - - - - - R W - > > With R being the current position of the read pointer, i.e. the > address of the first input item of the next B::work() call, > and W being the write pointer, i.e. the address of the first > output_item on the next A::work() call. > Each "- " (or "R " or "W ") is 1000 items worth of storage. > > Now, we can agree that in this buffer, there are thousand items that > must not be overwritten by the next A::work call, and 9000 items space > for new items. > > What GNU Radio does is employ memory mapping magic to allow A to > simply write contigously the next 9000 items; to the process, the > memory looks like this: > > - - - - - - - - R W -|- - - - - - - - R W - > > Notice that the second half is really just a transparent image of the > first, being inserted there by the memory control unit of your CPU. > > Now, that is awesome on so many levels: > * it allows developers to always write their applications like the in- > and output was just linear in memory, no matter where in their buffer > they are > * it allows full usage of the buffer, without having extra space allocated > > Of course, this has an architectural downside: > On any fully-fledged general purpose CPU platform I know, you can only > do this with pages; a page is simply the smallest amount of memory you > can tell your memory management unit to map somewhere. > On Linux, these pages are generally 4KB. That's usually a bit handy, > because it's a power of two bytes, which means that the typical item > sizes (1B for a byte/char, 2B for a short, 4B for a float or int, and > 8B for a std::complex<float> == gr_complex) fit well here, but not so > cool if your item's size is not a divider of 2**12. In that case, the > scheduler has no chance but to find the smallest common multiple of > 4096 and your item size. > > Greetings, > Marcus > > On 04/21/2015 08:04 PM, Anderson, Douglas J. wrote: >> Would it be possible to dive into this a bit deeper? I'm trying to >> get more familiar with the the scheduler and how the buffer is laid out. >> >> So using "tried to allocate 41 items of size 1592. Due to alignment >> requirements 512 were allocated" as an example, the scheduler looked >> at the requirements of the various blocks (or does each block have a >> dedicated buffer?) and decided it needed buffers of at least 41x1592 >> bytes? But assuming no block explicitly called set_alignment(), the >> scheduler uses the default page size (4096 bytes) and just allocated >> a number of 1592-byte blocks such that (n * 1592) % 4096 == 0? >> >> Am I on the right track with this? >> -Doug >> ------------------------------------------------------------------------ >> *From:* discuss-gnuradio-bounces+danderson=its.bldrdoc....@gnu.org >> [discuss-gnuradio-bounces+danderson=its.bldrdoc....@gnu.org] on >> behalf of Ali Riaz [ariaz.1...@gmail.com] >> *Sent:* Monday, April 20, 2015 12:16 PM >> *To:* discuss-gnuradio@gnu.org >> *Subject:* [Discuss-gnuradio] gr::buffer::allocate_buffer: warning >> >> Hello everyone, >> >> I'm getting the following warnings when running my application, does >> anyone know what this means? >> >> gr::buffer::allocate_buffer: warning: tried to allocate >> 41 items of size 1592. Due to alignment requirements >> 512 were allocated. If this isn't OK, consider padding >> your structure to a power-of-two bytes. >> On this platform, our allocation granularity is 4096 bytes. >> gr::buffer::allocate_buffer: warning: tried to allocate >> 82 items of size 796. Due to alignment requirements >> 1024 were allocated. If this isn't OK, consider padding >> your structure to a power-of-two bytes. >> On this platform, our allocation granularity is 4096 bytes. >> gr::buffer::allocate_buffer: warning: tried to allocate >> 82 items of size 796. Due to alignment requirements >> 1024 were allocated. If this isn't OK, consider padding >> your structure to a power-of-two bytes. >> On this platform, our allocation granularity is 4096 bytes. >> gr::buffer::allocate_buffer: warning: tried to allocate >> 82 items of size 796. Due to alignment requirements >> 1024 were allocated. If this isn't OK, consider padding >> your structure to a power-of-two bytes. >> On this platform, our allocation granularity is 4096 bytes >> >> Best, >> Ali >> >> >> _______________________________________________ >> Discuss-gnuradio mailing list >> Discuss-gnuradio@gnu.org >> https://lists.gnu.org/mailman/listinfo/discuss-gnuradio >
_______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org https://lists.gnu.org/mailman/listinfo/discuss-gnuradio
