On Saturday, 27 May 2017 14:05:11 UTC+3, Chun Zhang wrote: > > Thanks Kevin and Egon! > > With a few experiments, I found that the logging, even to a file, is quite > time consuming, so turning off logging helps, resulting in 500Mbps-ish no > drop rate; however, still not even close to Gbps. > > Then I turned on both lazy and nocopy option in the decoding option, the > lazy options seems to help. I got something close to 700Mbps, where the > sender's limit is reached. > > Given that said, the program does not nothing but receiving the packet at > this moment. Any actual processing of the packet in the same thread > significantly hurt the rate. Besides spinning multiple thread to handle the > actual work, anything else in the gopacket land that can be done? >
Profile your code. :) > > Thanks again! > Chun > > > > On Saturday, May 27, 2017 at 5:24:13 AM UTC-4, Kevin Conway wrote: >> >> > can only handle up to 250Mbps-ish traffic >> >> I'm not familiar with gopacket, but I have seen multiple occasions where >> logging to a file or stdout became a bottleneck. Your code snippet is >> logging on every packet which seems excessive. Try logging with less >> frequency and, if using stdout, consider using a log destination with >> different buffering characteristics like a file or syslog over UDP. >> >> On Fri, May 26, 2017 at 3:59 PM Egon <egon...@gmail.com> wrote: >> >>> On Friday, 26 May 2017 20:51:55 UTC+3, Chun Zhang wrote: >>>> >>>> Good point. >>>> as a comparison: tcpdump -w /dev/null can handle up to 750Mbps, where >>>> sending machine's speed limit reached. I think it should be able to >>>> handle >>>> line rate. >>>> >>>> Are those two packages lighter/faster than gopacket? >>>> >>> >>> Nevermind, just noticed... gopacket/pcap is a fork of akrennmair/gopcap >>> >>> Anyways, to get more information on what is taking time in your program >>> see https://blog.golang.org/profiling-go-programs >>> >>> Maybe try something like this: >>> >>> handle, err := pcap.OpenLive(device, snapshot_len, promiscuous, timeout) >>> >>> // ... >>> for { >>> data, ci, err := handle.ZeroCopyReadPacketData() >>> // ... >>> >>> >>> This should remove allocations from critical path. >>> >>> *PS: code untested and may contain typos :P* >>> >>> >>>> >>>> >>>> Thanks, >>>> Chun >>>> >>>> On Friday, May 26, 2017 at 12:37:55 PM UTC-4, Egon wrote: >>>>> >>>>> As a baseline measurement I suggest writing the same code in C; this >>>>> shows how much your VM / config / machine can handle. >>>>> >>>>> With gopacket -- use src.NextPacket instead of Packets. >>>>> >>>>> There are also: https://github.com/akrennmair/gopcap and >>>>> https://github.com/miekg/pcap >>>>> >>>>> + Egon >>>>> >>>>> On Friday, 26 May 2017 19:01:20 UTC+3, Chun Zhang wrote: >>>>>> >>>>>> Hi, All, >>>>>> >>>>>> I am trying to write a small program to handle packets coming from a >>>>>> GigE wire. The testing code snip is as below. >>>>>> >>>>>> The problem I am facing is that this is extremely slow, can only >>>>>> handle up to 250Mbps-ish traffic with normal ipv4 sized packets, >>>>>> anything >>>>>> above that resulting significant packet drop. Note that I am doing >>>>>> nothing >>>>>> with the packet at this moment. If I try to do any packet processing, >>>>>> then >>>>>> apparently it gets slower. >>>>>> >>>>>> Has anybody measured the efficiency of the gopacket package? Is there >>>>>> any other faster alternatives? >>>>>> >>>>>> PS: the host machine is an ubuntu VM with 12-core and 12G memory, but >>>>>> looks only 2 cores are used for this program. >>>>>> >>>>>> Thanks, >>>>>> Chun >>>>>> >>>>>> >>>>>> >>>>>> // Open device >>>>>> handle, err = pcap.OpenLive(device, snapshot_len, promiscuous, timeout) >>>>>> if err == nil { >>>>>> Info.Println("Open interface ", device, "successfully") >>>>>> >>>>>> } >>>>>> defer handle.Close() >>>>>> >>>>>> >>>>>> //fmt.Println("In the deafult reading case ", time.Now()) >>>>>> // Use the handle as a packet source to process all packets >>>>>> packetSource := gopacket.NewPacketSource(handle, >>>>>> handle.LinkType()) >>>>>> Info.Println("pcketsourc is ", packetSource, time.Now()) >>>>>> for packet := range packetSource.Packets() { >>>>>> >>>>>> Debug.Println("-------------------------------------------------------------------") >>>>>> count++ >>>>>> Warning.Println("packet count ", count) >>>>>> >>>>>> // write to a pcap for testing >>>>>> /*err = w.WritePacket(packet.Metadata().CaptureInfo, >>>>>> packet.Data()) >>>>>> if err != nil { >>>>>> fmt.Println(err) >>>>>> }*/ >>>>>> >>>>>> continue >>>>>> >>>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
