Re: [go-nuts] Performance of using splice(2) for TCP to File case

2024-12-04 Thread tokers
> Normally the pipe size doesn’t need to be very large if the producer and consumer “costs” are roughly the same. If the producer is costlier than the consumer than a large buffer does nothing. If the consumer is costlier than a large buffer only allow the producer to complete sooner. Otherwise,

Re: [go-nuts] Performance of using splice(2) for TCP to File case

2024-12-04 Thread Robert Engels
Normally the pipe size doesn’t need to be very large if the producer and consumer “costs” are roughly the same. If the producer is costlier than the consumer than a large buffer does nothing. If the consumer is costlier than a large buffer only allow the producer to complete sooner. Otherwise, y

Re: [go-nuts] Performance of using splice(2) for TCP to File case

2024-12-04 Thread Chao Zhang
Hi, > That sets the upper limit on how much data a kernel pipe can buffer to 2.5 > GiB. That is not reasonable. It is also an upper limit, not the default size. > Also, that kernel parameter only existed, AFAIK, in Linux 2.6.34 which was > released in 2010. Are you really using a Linux kernel t

Re: [go-nuts] Performance of using splice(2) for TCP to File case

2024-12-04 Thread Kurtis Rader
On Wed, Dec 4, 2024 at 12:33 AM tokers wrote: > I noticed (*os.File).ReadFrom has supported splice(2) since go/1.21, and > I'm trying to introduce it to my project. But the performance is not as > expected, which is slower than the normal io.Copy (buffer size = 1MB). Does > anyone who knows the r

[go-nuts] Performance of using splice(2) for TCP to File case

2024-12-04 Thread tokers
Hi, I noticed (*os.File).ReadFrom has supported splice(2) since go/1.21, and I'm trying to introduce it to my project. But the performance is not as expected, which is slower than the normal io.Copy (buffer size = 1MB). Does anyone who knows the reason? For using splice(2), I tweaked the param