Emanuele,
Here is ntopng.conf
-G=/var/run/ntopng.pid
-i=enp2s0
-m=10.12.17.0/24
-S=local
I do see unidirectional flows in flows_stats.lua for DNS. Incidentally, I
do also see alerts w/ non-zero replies (though most alerts are 0):
Host pihole has sent 211 DNS requests but received 7 DNS replies
I tried 2 different 30 minute PCAP files. In both cases, right at the 10
minute mark, I got alerts. How can I get these PCAP files to you?
Thanks,
Aaron
On Tue, May 12, 2020 at 4:13 AM Emanuele Faranda <fara...@ntop.org> wrote:
> Hi Aaron,
>
> Please see below.
> On 5/11/20 9:29 PM, Aaron Scamehorn wrote:
>
> Hi Emanuele,
>
> Thank you again for the detailed responses.
>
> From the interfaces page, I see these stats:
> Total Traffic 91.6 GB [103,062,265 Pkts] Dropped Packets 0 Pkts
> I don't see any dropped packets on the NIC either:
> ethtool -S enp2s0
> NIC statistics:
> tx_packets: 0
> rx_packets: 106581943
> tx_errors: 0
> rx_errors: 0
> rx_missed: 0
> align_errors: 0
> tx_single_collisions: 0
> tx_multi_collisions: 0
> unicast: 105432876
> broadcast: 350738
> multicast: 1149060
> tx_aborted: 0
> tx_underrun: 0
>
> As of right now, 2 of the hosts we are discussing are still in alert, at
> the original Date/Time of 07:25:01, and Duration is now "3 Days, 08:06:59".
>
> Given that my replies vs requests ratio is still configured at 50%, this
> means that, at every 5 minute interval for the last 3 Days, 8 hours, said
> host is receiving < 50% DNS replies, correct? I find this difficult to
> believe, and cannot find ANY missing packets in my pcap file.
>
> I have captured a 30 minute pcap file captured with this command:
> tcpdump -i enp2s0 -G 1800 -w /tmp/enp2s0.%FT%T.pcap host edgemax and port
> 53
>
> This file contains DNS traffic to/from edgemax only.
> I can count responses like this:
> tshark -t a -r enp2s0.2020-05-11T13:00:02.pcap | grep -c "Standard query
> response"
> 349
> And queries like this:
> tshark -t a -r enp2s0.2020-05-11T13:00:02.pcap | grep -c "Standard query
> 0x"
> 349
>
> In other words, no missing DNS responses in the 30 minutes spanning
> 13:00:02 to 13:29:51.
>
> I would think that the alert should "clear" because the threshold is not
> exceeded within that 30 minute pcap file.
>
> In any case, at 13:23, I manually click on the "Release" button for that
> alert. 2 minutes later, at 13:25:00, I receive this alert:
> Host edgemax has received 62 DNS requests but sent 0 DNS replies [5
> Minutes ratio: 0%]
>
> As stated previously, no missing DNS responses in the 30 minutes spanning
> 13:00:02 to 13:29:51. Why does ntopng think 62 replies are missing?
>
> Please report your ntopng.conf. If you look at the active ntopng DNS
> flows, can you identify unidirectional flows? You can also try to run
> ntopng on the PCAP file (--original-speed -i file.pcap). If you can
> reproduce using the PCAP file, please send it to me privately so that I can
> troubleshoot the problem.
>
>
> I exported 10 minutes of PCAP from if_stats.lua. Using the filter
> "(ip.dst_host == "10.12.17.1" or ip.src_host == "10.12.17.1") and dns" I am
> not able to find any missing DNS responses in wireshark. Interestingly, If
> I specify a BPF Filter ("port 53"), the downloaded PCAP file seems to only
> have 1 side (ie. edgemax is only a source, never a dest. Without a BPF
> Filter, the download is fine.
>
> This is probably a bug, please open an issue at
> https://github.com/ntop/ntopng .
>
> Regards,
>
> Emanuele
>
>
>
>
>
> On Mon, May 11, 2020 at 8:59 AM Emanuele Faranda <fara...@ntop.org> wrote:
>
>> Hi Aaron,
>>
>> Please see below:
>> On 5/8/20 10:27 PM, Aaron Scamehorn wrote:
>>
>> Thank you for your response. In the screenshot below, can you please
>> explain the significance of the "Date/Time" and the "Duration" columns?
>> What do they mean in this context?
>>
>> Date/Time: the time when the alert was triggered. Ntopng performs
>> periodic checks in order to trigger alerts. In this particular case, the
>> check on the requests/reply ratio is performed every 5 minutes. So this
>> means that problem started between 07:20 and 07:25 .
>>
>> Duration: the total time in which the problem was active. Again, the
>> check is performed every 5 minutes for this alert so 5 minutes is the
>> granularity.
>>
>>
>> Do I understand correctly that all 3 hosts triggered the alert at
>> 07:25:01 (OR 07:30:01) this morning? And that all three alerts are active
>> for the past 07:28:53 hours? Does this mean that there have been no new
>> additional DNS Reply/Request issues have been detected?
>>
>> As explained above, the problem started between 07:20 and 07:25 . For
>> 07:28:53 hours the problem was active on all the three hosts (the
>> requests/reply ratio threshold was exceeded for 07:28:53 hours).
>>
>>
>> I notice in "Past Alerts" tab, that there are many Reply/Request Alerts
>> for the same host with very short durations (screen shot #2). When/how
>> does an alert move from the "Engaged" to "Past" tab?
>>
>> In this case, the engaged alert becomes "past" alert when, after the
>> check performed every 5 minutes, the requests/reply ratio threshold is not
>> exceed anymore. This can happen as soon as the next check is performed (5
>> minutes).
>>
>>
>> So in the 2nd screenshot, fire-TV had an alert at 06:20:00 for 05:00
>> minutes where 18 requests received 0 replies. Then another alert at
>> 06:50:00 for 05:00 minutes. Were the 18 replies from the first alert
>> ultimately received? And they were received 5 minutes the alert occurred?
>>
>> The check is performed on the DNS packet counters. A DNS request cannot
>> take 5 minutes to be replied. The fact that the alert was closed after 5/10
>> minutes could be related to one of these events:
>>
>> - The host went idle
>>
>> - The host did not send enough DNS requests
>>
>> - The new DNS requests made by the host were successfully replied.
>>
>>
>> Context here is that 99% of the traffic is Internet traffic. Almost all
>> of the pihole traffic is to forwarders. BTW, the way pihole works (by
>> default) is it replies 0.0.0.0 for blocked hosts. It should respond to
>> every query.
>>
>> I tried the live_pcap_download.html
>> <https://www.ntop.org/guides/ntopng/advanced_features/live_pcap_download.html>
>> lua, but couldn't figure out the bpf_filter:
>> curl --cookie "user=admin; password=xxxxx" "
>> http://10.12.17.25:3000/lua/live_traffic.lua?ifid=0&duration=600&bpf_filter=\"port
>> 53\""
>>
>> I also tried the download pcap on the if_stats.lua page. The downloaded
>> pcap file seems to only contain incoming data (see wireshark)?
>>
>> This is consistent with the above alerts, please ensure that ntopng is
>> not dropping packets as this would explain this behavior.
>>
>>
>> If I just do a tshark on the same interface that ntopng is listening on,
>> I see all of the expected DNS query & replies. I am not able to correlate
>> the alerts to any missing packets.
>>
>> See response above.
>>
>> Regards,
>>
>> Emanuele
>>
>>
>>
>>
>> On Fri, May 8, 2020 at 2:53 AM Emanuele Faranda <fara...@ntop.org> wrote:
>>
>>> Hi Aaron,
>>>
>>> The alerts that you are reporting basically tell you that such hosts
>>> receive DNS requests but do not send a reply. In order to troubleshoot
>>> possible problems you should augment such information with the knowledge of
>>> your network.
>>>
>>> The first question to answer is, are that hosts expected to accept DNS
>>> requests? If not, are the requests generated from the internet or from the
>>> LAN? In the first case a firewall to block such DNS requests may be a good
>>> idea . In the latter case some hosts in the LAN may be misconfigured. In
>>> case of the pihole hosts, I expect pihole to block some DNS requests for
>>> advertisement sites so this could be a normal behaviour. The following
>>> ntopng features may also help you:
>>>
>>>
>>> https://www.ntop.org/guides/ntopng/advanced_features/live_pcap_download.html
>>>
>>>
>>> https://www.ntop.org/guides/ntopng/using_with_other_tools/n2disk.html
>>>
>>> https://www.ntop.org/guides/ntopng/historical_flows.html
>>>
>>> Regards,
>>> Emanuele
>>> On 5/7/20 5:57 PM, Aaron Scamehorn wrote:
>>>
>>> Hello,
>>>
>>> I'm trying to understand how/why I am getting the "Replies / Requests
>>> Ratio" warnings for DNS.
>>>
>>> I am suspect of these alerts, and would like to know how/why they are
>>> being generated. I am suspect for for the following reasons: 1) If it
>>> really is as bad as indicated, I should notice problems. 2) the "events'
>>> occur immediately after I clear the alerts, and tend to persist for hours.
>>>
>>> In any case, I cleared the alerts last night, and this is what they look
>>> like:
>>>
>>> 06/05/2020 22:15:00 12:31:28 Warning Replies / Requests Ratio Host
>>> edgemax.example.net
>>> <http://xps-630i.scamlan.net:3000/lua/host_details.lua?ifid=2&host=10.12.17.1@1&page=historical&epoch_begin=1588864588&epoch_end=1588868188>
>>> has received 54 DNS requests but sent 0 DNS replies [5 Minutes ratio: 0%]
>>>
>>> 06/05/2020 22:15:00 12:31:28 Warning Replies / Requests Ratio Host
>>> pihole.example.net
>>> <http://xps-630i.scamlan.net:3000/lua/host_details.lua?ifid=2&host=10.12.17.3@1&page=historical&epoch_begin=1588864588&epoch_end=1588868188>
>>> has sent 93 DNS requests but received 3 DNS replies [5 Minutes ratio: 3.2%]
>>>
>>> 06/05/2020 22:15:00 12:31:28 Warning Replies / Requests Ratio Host
>>> pihole-2.example.net
>>> <http://xps-630i.scamlan.net:3000/lua/host_details.lua?ifid=2&host=10.12.17.4@1&page=historical&epoch_begin=1588864588&epoch_end=1588868188>
>>> has sent 97 DNS requests but received 1 DNS reply [5 Minutes ratio: 1.0%]
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Ntop mailing
>>> listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>>
>>> _______________________________________________
>>> Ntop mailing list
>>> Ntop@listgateway.unipi.it
>>> http://listgateway.unipi.it/mailman/listinfo/ntop
>>
>>
>> _______________________________________________
>> Ntop mailing
>> listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>>
>> _______________________________________________
>> Ntop mailing list
>> Ntop@listgateway.unipi.it
>> http://listgateway.unipi.it/mailman/listinfo/ntop
>
>
> _______________________________________________
> Ntop mailing
> listNtop@listgateway.unipi.ithttp://listgateway.unipi.it/mailman/listinfo/ntop
>
> _______________________________________________
> Ntop mailing list
> Ntop@listgateway.unipi.it
> http://listgateway.unipi.it/mailman/listinfo/ntop
_______________________________________________
Ntop mailing list
Ntop@listgateway.unipi.it
http://listgateway.unipi.it/mailman/listinfo/ntop
