If you are really interested in how the Go code relates to the underlying
socket system calls, the code is readily available. AFAICT, the
TCPConn.Read() call *on linux* eventually comes down to the unix version
poll.PD.Read(). See https://golang.org/src/internal/poll/fd_unix.go#L145.
Before that is mostly just basic setup and then code to wrap the errors.
Unless you are on an embedded system or something, you will likely not find
that memory usage is the real bottleneck in practice, assuming you choose
reasonable constants and a reasonable number of workers. If GC thrashing is
a problem, you can always use a strategy to reuse the buffers.
Good Luck.
On Tuesday, December 31, 2019 at 10:22:45 AM UTC-5, Ron Wahler wrote:
>
> Thanks for all the great responses.
>
> How much of the read behavior is in the golang underlying code on the read
> and how much is
> on the underlying OS driver that implements the behavior of the read. I
> understand the stream nature of the TCP connection and I handle that in my
> code, I was just looking to optimize the buffer allocation for the read on
> each packet of the stream so I don't have to over allocate the buf if I
> scaled out my function to be concurrent. From the responses so far I think
> I just have to pick a buffer size for the Read() or the ReadAtLeast() and
> just process multiple buffers with my pre-allocated guess on the buffer
> size as I am already doing.
>
> I was also looking to understand the read implementation of GoLand and how
> it utilizes the underlying drivers, so any info on that or where to read
> the code for golang integration with the drivers would be great.
>
> cheers,
> Ron
>
> On Friday, December 27, 2019 at 5:11:42 PM UTC-7, Ron Wahler wrote:
>>
>> I am looking for a net.conn standard read that would return a data buffer
>> the exact size of the read. I am trying to read an unknown amount of byte
>> data from the connection. With the read i am using I am required to
>> pre-allocate a buffer and pass that buffer to the read. I am looking for a
>> read that works more like the ReadString , but is for a byte slice.
>>
>> // I want something similar to this read that returns the read string
>> into the message string.
>>
>> message, err := bufio.NewReader(ServerConn).ReadString('\n')
>>
>> if ( err != nil ){
>>
>> fmt.Println("RELAY: ERROR: Reg Message read
>> err:", err)
>>
>> return
>>
>> }
>>
>>
>>
>> // had to preallocate a buffer, but I want a read to return me a buffer
>> so I don't have to guess how big to make it.
>>
>> buf := make([]byte, 1024*32)
>>
>> // READ FROM CLIENT
>>
>> nBytes, err := Csrc.Read(buf)
>>
>>
>>
>> Is this not possible, I have not seen any examples that would indicate
>> that there is a standard library that would do something like what I am
>> looking for.
>>
>>
>> thanks,
>>
>> Ron
>>
>
--
You received this message because you are subscribed to the Google Groups
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/golang-nuts/7cf5b2f3-b8e9-425c-ae0e-7391a2b3158d%40googlegroups.com.
