Hi, All, Update on this issue. Based on the suggestion I got earlier, I dedicated one thread, which is locked to a os thread to handle packet receiving, this thread then put the received packet on a buffered channel/queue. Without doing any extra work, this thread is able to take packets up to 800Mbps-ish, which is the limit of the sender.
12 goroutines are then kicked off to take items from this queue and distribute them to other work queues for further processing. The distributing thread parses the packet to a gopacket.Packet interface and hashes based on the ip address etc. Even though I have used the faster version, the parse routine takes a LOT of the cpu power. By profiling the program, it seems the parsing takes 1/3 of whole time. The whole app is then limited to roughly 120Mbps, aka, 50kpps. The decoding routine I am using is pretty much like the example here Decoding Packets Faster http://www.devdungeon.com/content/packet-capture-injection-and-analysis-gopacket I am wondering what further optimization can I do to speed this up? Thanks, Chun On Tuesday, May 30, 2017 at 10:50:50 AM UTC-4, Chun Zhang wrote: > > Thank you Rajanikanth, Kevin and Egon! I will explore the ideas you guys > provided and keep you updated. > > Best Regards, > Chun > > On Sunday, May 28, 2017 at 2:25:03 AM UTC-4, Egon wrote: >> >> >> >> On Saturday, 27 May 2017 14:05:11 UTC+3, Chun Zhang wrote: >>> >>> Thanks Kevin and Egon! >>> >>> With a few experiments, I found that the logging, even to a file, is >>> quite time consuming, so turning off logging helps, resulting in >>> 500Mbps-ish no drop rate; however, still not even close to Gbps. >>> >>> Then I turned on both lazy and nocopy option in the decoding option, the >>> lazy options seems to help. I got something close to 700Mbps, where the >>> sender's limit is reached. >>> >>> Given that said, the program does not nothing but receiving the packet >>> at this moment. Any actual processing of the packet in the same thread >>> significantly hurt the rate. Besides spinning multiple thread to handle the >>> actual work, anything else in the gopacket land that can be done? >>> >> >> Profile your code. :) >> >> >>> >>> Thanks again! >>> Chun >>> >>> >>> >>> On Saturday, May 27, 2017 at 5:24:13 AM UTC-4, Kevin Conway wrote: >>>> >>>> > can only handle up to 250Mbps-ish traffic >>>> >>>> I'm not familiar with gopacket, but I have seen multiple occasions >>>> where logging to a file or stdout became a bottleneck. Your code snippet >>>> is >>>> logging on every packet which seems excessive. Try logging with less >>>> frequency and, if using stdout, consider using a log destination with >>>> different buffering characteristics like a file or syslog over UDP. >>>> >>>> On Fri, May 26, 2017 at 3:59 PM Egon <egon...@gmail.com> wrote: >>>> >>>>> On Friday, 26 May 2017 20:51:55 UTC+3, Chun Zhang wrote: >>>>>> >>>>>> Good point. >>>>>> as a comparison: tcpdump -w /dev/null can handle up to 750Mbps, where >>>>>> sending machine's speed limit reached. I think it should be able to >>>>>> handle >>>>>> line rate. >>>>>> >>>>>> Are those two packages lighter/faster than gopacket? >>>>>> >>>>> >>>>> Nevermind, just noticed... gopacket/pcap is a fork of akrennmair/gopcap >>>>> >>>>> Anyways, to get more information on what is taking time in your >>>>> program see https://blog.golang.org/profiling-go-programs >>>>> >>>>> Maybe try something like this: >>>>> >>>>> handle, err := pcap.OpenLive(device, snapshot_len, promiscuous, >>>>> timeout) >>>>> >>>>> // ... >>>>> for { >>>>> data, ci, err := handle.ZeroCopyReadPacketData() >>>>> // ... >>>>> >>>>> >>>>> This should remove allocations from critical path. >>>>> >>>>> *PS: code untested and may contain typos :P* >>>>> >>>>> >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Chun >>>>>> >>>>>> On Friday, May 26, 2017 at 12:37:55 PM UTC-4, Egon wrote: >>>>>>> >>>>>>> As a baseline measurement I suggest writing the same code in C; this >>>>>>> shows how much your VM / config / machine can handle. >>>>>>> >>>>>>> With gopacket -- use src.NextPacket instead of Packets. >>>>>>> >>>>>>> There are also: https://github.com/akrennmair/gopcap and >>>>>>> https://github.com/miekg/pcap >>>>>>> >>>>>>> + Egon >>>>>>> >>>>>>> On Friday, 26 May 2017 19:01:20 UTC+3, Chun Zhang wrote: >>>>>>>> >>>>>>>> Hi, All, >>>>>>>> >>>>>>>> I am trying to write a small program to handle packets coming from >>>>>>>> a GigE wire. The testing code snip is as below. >>>>>>>> >>>>>>>> The problem I am facing is that this is extremely slow, can only >>>>>>>> handle up to 250Mbps-ish traffic with normal ipv4 sized packets, >>>>>>>> anything >>>>>>>> above that resulting significant packet drop. Note that I am doing >>>>>>>> nothing >>>>>>>> with the packet at this moment. If I try to do any packet processing, >>>>>>>> then >>>>>>>> apparently it gets slower. >>>>>>>> >>>>>>>> Has anybody measured the efficiency of the gopacket package? Is >>>>>>>> there any other faster alternatives? >>>>>>>> >>>>>>>> PS: the host machine is an ubuntu VM with 12-core and 12G memory, >>>>>>>> but looks only 2 cores are used for this program. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Chun >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> // Open device >>>>>>>> handle, err = pcap.OpenLive(device, snapshot_len, promiscuous, timeout) >>>>>>>> if err == nil { >>>>>>>> Info.Println("Open interface ", device, "successfully") >>>>>>>> >>>>>>>> } >>>>>>>> defer handle.Close() >>>>>>>> >>>>>>>> >>>>>>>> //fmt.Println("In the deafult reading case ", time.Now()) >>>>>>>> // Use the handle as a packet source to process all packets >>>>>>>> packetSource := gopacket.NewPacketSource(handle, >>>>>>>> handle.LinkType()) >>>>>>>> Info.Println("pcketsourc is ", packetSource, time.Now()) >>>>>>>> for packet := range packetSource.Packets() { >>>>>>>> >>>>>>>> Debug.Println("-------------------------------------------------------------------") >>>>>>>> count++ >>>>>>>> Warning.Println("packet count ", count) >>>>>>>> >>>>>>>> // write to a pcap for testing >>>>>>>> /*err = w.WritePacket(packet.Metadata().CaptureInfo, >>>>>>>> packet.Data()) >>>>>>>> if err != nil { >>>>>>>> fmt.Println(err) >>>>>>>> }*/ >>>>>>>> >>>>>>>> continue >>>>>>>> >>>>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "golang-nuts" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to golang-nuts...@googlegroups.com. >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
