Hi Paolo, thanks, I decided to build pmacct from scratch and the problem went away. Thank you
We are having a different problem now with kafka/avro and the encoding. I will start a new email thread and keep it separated. --Jaime On Sun, Mar 18, 2018 at 3:55 AM, Paolo Lucente <[email protected]> wrote: > > Hi Jaime, > > Is it possible you are launching a different nfacctd executable (since > you are not using the path), maybe in /usr/local/sbin? The CL switches > you pass to the configure script do not match with those compiled-in the > executable you launch, ie.: > > INFO ( default/core ): NetFlow Accounting Daemon, nfacctd 1.7.1-git > (20180315-00) > INFO ( default/core ): '--enable-mysql' '--enable-kafka' > '--enable-jansson' '--enable-l2' '--enable-ipv6' '--enable-64bit' > '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' > '--enable-st-bins' > > You are launching an executable compiled with --enable-mysql > --enable-kafka --enable-jansson. Does this ring a bell? > > Paolo > > On Sat, Mar 17, 2018 at 11:23:49AM -0700, Jaime Botello wrote: > > Hi, > > > > I'm new to this mailing list and my apologies if this is not the right > > place to discuss this. > > > > we are trying to build pmacct netflow collector with avro/kafka support. > > > > After meeting all requirements, avro, kafka among other I was finally > able > > to run ./configure without errors > > > > ubuntu@pmacct:~/pmacct$ ./configure --enable-avro --enable-kafka > > --enable-jansson > > > > checking whether to enable Avro support... yes > > checking for AVRO... yes > > checking for avro_record_get in -lavro... yes > > checking avro.h usability... yes > > checking avro.h presence... yes > > checking for avro.h... yes > > > > > > ubuntu@ip-10-180-221-47:~/pmacct$ nfacctd -f netflow_kafka.conf -d > > DEBUG: [netflow_kafka.conf] plugin name/type: 'default'/'core'. > > DEBUG: [netflow_kafka.conf] plugin name/type: 'default_kafka'/'kafka'. > > DEBUG: [netflow_kafka.conf] aggregate:src_host, dst_host, src_port, > > dst_port, proto, tos, src_as, dst_as, peer_src_ip, peer_dst_ip, in_iface, > > out_iface, src_net, dst_net, src_mask, dst_mask, tcpflags, sampling_rate, > > timestamp_start, timestamp_end, timestamp_arrival > > DEBUG: [netflow_kafka.conf] nfacctd_port:2055 > > DEBUG: [netflow_kafka.conf] nfacctd_ip:10.180.222.225 > > DEBUG: [netflow_kafka.conf] kafka_output:avro > > *WARN: [netflow_kafka.conf] 'message_broker_output' set to avro but will > > produce no output (missing --enable-avro).* > > DEBUG: [netflow_kafka.conf] kafka_topic:netflow > > DEBUG: [netflow_kafka.conf] kafka_refresh_time:10 > > DEBUG: [netflow_kafka.conf] kafka_history:1m > > DEBUG: [netflow_kafka.conf] kafka_history_roundoff:m > > DEBUG: [netflow_kafka.conf] kafka_broker_host:10.180.221.152 > > DEBUG: [netflow_kafka.conf] kafka_broker_port:9092 > > DEBUG: [netflow_kafka.conf] debug:true > > INFO ( default/core ): NetFlow Accounting Daemon, nfacctd 1.7.1-git > > (20180315-00) > > INFO ( default/core ): '--enable-mysql' '--enable-kafka' > > '--enable-jansson' '--enable-l2' '--enable-ipv6' '--enable-64bit' > > '--enable-traffic-bins' '--enable-bgp-bins' '--enable-bmp-bins' > > '--enable-st-bins' > > INFO ( default/core ): Reading configuration file > > '/home/ubuntu/pmacct/netflow_kafka.conf'. > > INFO ( default_kafka/kafka ): plugin_pipe_size=4096000 bytes > > plugin_buffer_size=456 bytes > > INFO ( default_kafka/kafka ): ctrl channel: obtained=212992 bytes > > target=71856 bytes > > > > > > After a while and when pmacct is ready to create output, it throws the > > following message > > > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > DEBUG ( default_kafka/kafka ): compose_avro(): AVRO object not created > due > > to missing --enable-avro > > > > This is my config file for reference > > > > ubuntu@pmacct:~/pmacct$ more netflow_kafka.conf > > ! .. > > plugins: kafka > > ! > > aggregate: src_host, dst_host, src_port, dst_port, proto, tos, src_as, > > dst_as, peer_src_ip, peer_dst_ip, in_iface, out_iface, src_net, dst_net, > > src_mask, dst_mask, tcpflags, sampling_rate, timestamp_start, > > timestamp_end, timestamp_arrival > > ! > > nfacctd_port: 2055 > > nfacctd_ip: 10.180.222.225 > > ! > > ! > > kafka_output: avro > > kafka_topic: netflow > > kafka_refresh_time: 10 > > kafka_history: 1m > > kafka_history_roundoff: m > > kafka_broker_host: 10.180.221.152 > > kafka_broker_port: 9092 > > > > thank in advance for any assistance or pointers > > -- > > *Jaime Botello** (aka Jimbo) * > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists > -- *Jaime Botello** (aka Jimbo) *// *Riot Games* : Riot Direct // c: 310-210-9772 // summoner: Riot R3lick
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
