Thanks Kevin and Egon! With a few experiments, I found that the logging, even to a file, is quite time consuming, so turning off logging helps, resulting in 500Mbps-ish no drop rate; however, still not even close to Gbps.
Then I turned on both lazy and nocopy option in the decoding option, the lazy options seems to help. I got something close to 700Mbps, where the sender's limit is reached. Given that said, the program does not nothing but receiving the packet at this moment. Any actual processing of the packet in the same thread significantly hurt the rate. Besides spinning multiple thread to handle the actual work, anything else in the gopacket land that can be done? Thanks again! Chun On Saturday, May 27, 2017 at 5:24:13 AM UTC-4, Kevin Conway wrote: > > > can only handle up to 250Mbps-ish traffic > > I'm not familiar with gopacket, but I have seen multiple occasions where > logging to a file or stdout became a bottleneck. Your code snippet is > logging on every packet which seems excessive. Try logging with less > frequency and, if using stdout, consider using a log destination with > different buffering characteristics like a file or syslog over UDP. > > On Fri, May 26, 2017 at 3:59 PM Egon <egon...@gmail.com <javascript:>> > wrote: > >> On Friday, 26 May 2017 20:51:55 UTC+3, Chun Zhang wrote: >>> >>> Good point. >>> as a comparison: tcpdump -w /dev/null can handle up to 750Mbps, where >>> sending machine's speed limit reached. I think it should be able to handle >>> line rate. >>> >>> Are those two packages lighter/faster than gopacket? >>> >> >> Nevermind, just noticed... gopacket/pcap is a fork of akrennmair/gopcap >> >> Anyways, to get more information on what is taking time in your program >> see https://blog.golang.org/profiling-go-programs >> >> Maybe try something like this: >> >> handle, err := pcap.OpenLive(device, snapshot_len, promiscuous, timeout) >> >> // ... >> for { >> data, ci, err := handle.ZeroCopyReadPacketData() >> // ... >> >> >> This should remove allocations from critical path. >> >> *PS: code untested and may contain typos :P* >> >> >>> >>> >>> Thanks, >>> Chun >>> >>> On Friday, May 26, 2017 at 12:37:55 PM UTC-4, Egon wrote: >>>> >>>> As a baseline measurement I suggest writing the same code in C; this >>>> shows how much your VM / config / machine can handle. >>>> >>>> With gopacket -- use src.NextPacket instead of Packets. >>>> >>>> There are also: https://github.com/akrennmair/gopcap and >>>> https://github.com/miekg/pcap >>>> >>>> + Egon >>>> >>>> On Friday, 26 May 2017 19:01:20 UTC+3, Chun Zhang wrote: >>>>> >>>>> Hi, All, >>>>> >>>>> I am trying to write a small program to handle packets coming from a >>>>> GigE wire. The testing code snip is as below. >>>>> >>>>> The problem I am facing is that this is extremely slow, can only >>>>> handle up to 250Mbps-ish traffic with normal ipv4 sized packets, anything >>>>> above that resulting significant packet drop. Note that I am doing >>>>> nothing >>>>> with the packet at this moment. If I try to do any packet processing, >>>>> then >>>>> apparently it gets slower. >>>>> >>>>> Has anybody measured the efficiency of the gopacket package? Is there >>>>> any other faster alternatives? >>>>> >>>>> PS: the host machine is an ubuntu VM with 12-core and 12G memory, but >>>>> looks only 2 cores are used for this program. >>>>> >>>>> Thanks, >>>>> Chun >>>>> >>>>> >>>>> >>>>> // Open device >>>>> handle, err = pcap.OpenLive(device, snapshot_len, promiscuous, timeout) >>>>> if err == nil { >>>>> Info.Println("Open interface ", device, "successfully") >>>>> >>>>> } >>>>> defer handle.Close() >>>>> >>>>> >>>>> //fmt.Println("In the deafult reading case ", time.Now()) >>>>> // Use the handle as a packet source to process all packets >>>>> packetSource := gopacket.NewPacketSource(handle, handle.LinkType()) >>>>> Info.Println("pcketsourc is ", packetSource, time.Now()) >>>>> for packet := range packetSource.Packets() { >>>>> >>>>> Debug.Println("-------------------------------------------------------------------") >>>>> count++ >>>>> Warning.Println("packet count ", count) >>>>> >>>>> // write to a pcap for testing >>>>> /*err = w.WritePacket(packet.Metadata().CaptureInfo, >>>>> packet.Data()) >>>>> if err != nil { >>>>> fmt.Println(err) >>>>> }*/ >>>>> >>>>> continue >>>>> >>>>> -- >> You received this message because you are subscribed to the Google Groups >> "golang-nuts" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to golang-nuts...@googlegroups.com <javascript:>. >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
