Ok, that's strange. Yeah, the TSDB should also reject duplicate samples and out-of-order samples for the same series if they came directly from targets and not a recording rule. So I'm not sure what's happening here. Maybe you could share your prometheus.yml and a sample target.yml as well for good measure?
On Tue, Dec 13, 2022 at 1:21 PM Umut Cokbilir <[email protected]> wrote: > Thanks for your detail explanation. You can see my answers below. > > I am not doing federation or any sort of availability. > I've checked [1h] then [1d] and haven't seen any multiple data points :( > I am not scraping the same targets twice but i am updating target.yml > every hour. Does it affect? > i am not sure there is any bug in my exporter config but it scrapes data > from mydevices every 5 minutes. > Please find curl output attached. > > > 13 Aralık 2022 Salı tarihinde saat 14:44:31 UTC+3 itibarıyla Brian Candler > şunları yazdı: > >> The first error explicitly mentions a particular metric "fix:min_in", and >> the input to the recording rule which generates that is the metric >> "access:ifHCInOctets:FIX_rate" (rule 16, counting from zero) >> But it also mentions an error for the metric >> "access:ifHCInOctets:FIX_rate", which takes its input from "ifHCInOctets" >> (rule 4) >> >> So my guess is that you have duplicate data somewhere, posisbly in >> ifHCInOctets. Try these queries: >> >> >> ifHCInOctets{instance="10.85.4.3",ifAlias="FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26 >> ,330@-||10||"}[1h] >> >> access:ifHCInOctets:FIX_rate{instance="10.85.4.3",ifAlias="FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26 >> ,330@-||10||"}[1h] >> >> Do you see multiple data points with the same timestamp? In which >> timeseries? if not, try extending the time range to [1d] >> >> This would however require you to have multiple values of ifHCInOctets >> with the same value of all labels (same ifAlias, same ifDescr, same >> ifIndex). >> >> If that's what you're seeing then you have to work backwards: are you >> scraping the same target twice? Or is there a bug in your snmp_exporter >> config? Try scraping the device manually using curl: >> >> curl -g 'localhost:9115/snmp?module=XXXX&target=10.85.4.3' >> >> (Having said that, if there were duplicates at this point, I'd be >> expecting them to be rejected at scrape time). >> >> Are you doing any sort of high-availability or federation? If so, you >> need different external labels for each data source. >> >> On Tuesday, 13 December 2022 at 10:55:19 UTC [email protected] wrote: >> >>> Hi Julius, >>> >>> Please find my recording rules file attached. There isn't any rules >>> using same names. >>> Please also check it? Where i am doing wrong? >>> >>> Thanks >>> >>> 13 Aralık 2022 Salı tarihinde saat 12:07:31 UTC+3 itibarıyla >>> [email protected] şunları yazdı: >>> >>>> Hi Umut, >>>> >>>> This error occurs when something in Prometheus tries to append a sample >>>> to a time series with the same timestamp as the last already stored sample >>>> in the series, but with a different sample value. Usually this is because >>>> of some accidental collision between time series identities (metric name + >>>> label set). Is there any chance that anything else in your Prometheus setup >>>> also produces metrics with the same names as the ones in your recording >>>> rules? Maybe the recording rules are duplicated, but with different values >>>> (probably in the same rule group, since it's the same timestamp)? >>>> >>>> Regards, >>>> Julius >>>> >>>> On Tue, Dec 13, 2022 at 6:54 AM Umut Cokbilir <[email protected]> >>>> wrote: >>>> >>>>> Hi All, >>>>> >>>>> I couldn't understand the root cause of the problem below. How to >>>>> solve it? >>>>> >>>>> Prometheus version: prometheus-2.40.3.linux-amd64 >>>>> snmp_exporter version: snmp_exporter-0.20.0 >>>>> >>>>> Rules: >>>>> - record: fix:min_in >>>>> expr: min_over_time(access:ifHCInOctets:FIX_rate[1d]) >>>>> - record: fix:min_out >>>>> expr: min_over_time(access:ifHCOutOctets:FIX_rate[1d]) >>>>> - record: fix:max_in >>>>> expr: max_over_time(access:ifHCInOctets:FIX_rate[1d]) >>>>> - record: fix:max_out >>>>> expr: max_over_time(access:ifHCOutOctets:FIX_rate[1d]) >>>>> - record: fix:avg_in >>>>> expr: avg_over_time(access:ifHCInOctets:FIX_rate[1d]) >>>>> - record: fix:avg_out >>>>> expr: avg_over_time(access:ifHCOutOctets:FIX_rate[1d]) >>>>> - record: fix:avg_in:12h >>>>> expr: avg_over_time(access:ifHCInOctets:FIX_rate[12h]) >>>>> - record: fix:avg_out:12h >>>>> expr: avg_over_time(access:ifHCOutOctets:FIX_rate[12h]) >>>>> >>>>> 04:41:23 --ts=2022-12-13T01:41:23.771Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:min_in >>>>> index=16 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:min_in\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 22676.373333333333 @[1670895366553]" >>>>> 04:41:25 --ts=2022-12-13T01:41:25.099Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in >>>>> index=20 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 45519.09881820317 @[1670895366553]" >>>>> 04:41:25 --ts=2022-12-13T01:41:25.495Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out >>>>> index=21 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 571800.1416907905 @[1670895366553]" >>>>> 04:41:26 --ts=2022-12-13T01:41:25.864Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in:12h >>>>> index=22 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 37126.089073190036 @[1670895366553]" >>>>> 04:41:26 --ts=2022-12-13T01:41:26.208Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out:12h >>>>> index=23 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 568421.9923866978 @[1670895366553]" >>>>> 05:21:23 --ts=2022-12-13T02:21:23.715Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in >>>>> index=20 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 45353.12701316737 @[1670897766553]" >>>>> 05:21:24 --ts=2022-12-13T02:21:24.126Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out >>>>> index=21 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 571748.0096685543 @[1670897766553]" >>>>> 05:21:24 --ts=2022-12-13T02:21:24.453Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in:12h >>>>> index=22 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 31094.769935320466 @[1670897766553]" >>>>> 05:21:25 --ts=2022-12-13T02:21:24.794Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out:12h >>>>> index=23 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 567518.4603637857 @[1670897766553]" >>>>> 06:06:26 --ts=2022-12-13T03:06:25.984Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in >>>>> index=20 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 45189.166572681956 @[1670900466553]" >>>>> 06:06:26 --ts=2022-12-13T03:06:26.353Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out >>>>> index=21 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 571701.6636125795 @[1670900466553]" >>>>> 06:06:26 --ts=2022-12-13T03:06:26.679Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in:12h >>>>> index=22 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 29880.13207330078 @[1670900466553]" >>>>> 06:06:27 --ts=2022-12-13T03:06:27.010Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out:12h >>>>> index=23 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 567379.9435644317 @[1670900466553]" >>>>> 06:21:27 --ts=2022-12-13T03:21:27.089Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in >>>>> index=20 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 45110.958791205536 @[1670901366553]" >>>>> 06:21:27 --ts=2022-12-13T03:21:27.496Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out >>>>> index=21 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 571723.652811564 @[1670901366553]" >>>>> 06:21:27 --ts=2022-12-13T03:21:27.832Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in:12h >>>>> index=22 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 29745.59755724392 @[1670901366553]" >>>>> 06:21:28 --ts=2022-12-13T03:21:28.173Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out:12h >>>>> index=23 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 567638.6622438562 @[1670901366553]" >>>>> 06:56:28 --ts=2022-12-13T03:56:28.058Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in >>>>> index=20 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 44958.979497887565 @[1670903466553]" >>>>> 06:56:28 --ts=2022-12-13T03:56:28.650Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out >>>>> index=21 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 571641.2677371124 @[1670903466553]" >>>>> 06:56:29 --ts=2022-12-13T03:56:29.089Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in:12h >>>>> index=22 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 29316.18369902085 @[1670903466553]" >>>>> 06:56:29 --ts=2022-12-13T03:56:29.713Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out:12h >>>>> index=23 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 567614.2432218753 @[1670903466553]" >>>>> 07:06:08 --ts=2022-12-13T04:06:07.854Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw >>>>> name=access:ifHCInOctets:rate index=0 msg="Rule evaluation result >>>>> discarded" err="duplicate sample for timestamp" >>>>> sample="{__name__=\"access:ifHCInOctets:rate\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 26498.213333333337 @[1670904066553]" >>>>> 07:06:10 --ts=2022-12-13T04:06:09.708Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw >>>>> name=access:ifHCOutOctets:rate index=1 msg="Rule evaluation result >>>>> discarded" err="duplicate sample for timestamp" >>>>> sample="{__name__=\"access:ifHCOutOctets:rate\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 569222.2666666667 @[1670904066553]" >>>>> 07:06:12 --ts=2022-12-13T04:06:11.181Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw >>>>> name=access:ifHCInOctets:FIX_rate index=4 msg="Rule evaluation result >>>>> discarded" err="duplicate sample for timestamp" >>>>> sample="{__name__=\"access:ifHCInOctets:FIX_rate\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 26498.213333333337 @[1670904066553]" >>>>> 07:06:13 --ts=2022-12-13T04:06:11.676Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw >>>>> name=access:ifHCOutOctets:FIX_rate index=5 msg="Rule evaluation result >>>>> discarded" err="duplicate sample for timestamp" >>>>> sample="{__name__=\"access:ifHCOutOctets:FIX_rate\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 569222.2666666667 @[1670904066553]" >>>>> 07:51:27 --ts=2022-12-13T04:51:27.065Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in >>>>> index=20 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 44893.41348050907 @[1670906766553]" >>>>> 07:51:27 --ts=2022-12-13T04:51:27.564Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out >>>>> index=21 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 571623.9997716311 @[1670906766553]" >>>>> 07:51:28 --ts=2022-12-13T04:51:27.907Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in:12h >>>>> index=22 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 28273.031953867194 @[1670906766553]" >>>>> 07:51:28 --ts=2022-12-13T04:51:28.245Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out:12h >>>>> index=23 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 568057.166386401 @[1670906766553]" >>>>> 08:01:08 --ts=2022-12-13T05:01:08.140Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw >>>>> name=access:ifHCInOctets:rate index=0 msg="Rule evaluation result >>>>> discarded" err="duplicate sample for timestamp" >>>>> sample="{__name__=\"access:ifHCInOctets:rate\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 26605.573333333334 @[1670907366553]" >>>>> 08:01:10 --ts=2022-12-13T05:01:09.841Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw >>>>> name=access:ifHCOutOctets:rate index=1 msg="Rule evaluation result >>>>> discarded" err="duplicate sample for timestamp" >>>>> sample="{__name__=\"access:ifHCOutOctets:rate\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 566597.4400000001 @[1670907366553]" >>>>> 08:01:12 --ts=2022-12-13T05:01:11.039Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw >>>>> name=access:ifHCInOctets:FIX_rate index=4 msg="Rule evaluation result >>>>> discarded" err="duplicate sample for timestamp" >>>>> sample="{__name__=\"access:ifHCInOctets:FIX_rate\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 26605.573333333334 @[1670907366553]" >>>>> 08:01:13 --ts=2022-12-13T05:01:11.295Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw >>>>> name=access:ifHCOutOctets:FIX_rate index=5 msg="Rule evaluation result >>>>> discarded" err="duplicate sample for timestamp" >>>>> sample="{__name__=\"access:ifHCOutOctets:FIX_rate\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 566597.4400000001 @[1670907366553]" >>>>> 08:16:26 --ts=2022-12-13T05:16:26.233Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in >>>>> index=20 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 44828.267673529954 @[1670908266553]" >>>>> 08:16:26 --ts=2022-12-13T05:16:26.661Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out >>>>> index=21 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 571600.3475860697 @[1670908266553]" >>>>> 08:16:27 --ts=2022-12-13T05:16:27.003Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_in:12h >>>>> index=22 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_in:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 27422.084942333982 @[1670908266553]" >>>>> 08:16:27 --ts=2022-12-13T05:16:27.338Z caller=manager.go:684 >>>>> level=warn component="rule manager" >>>>> file=/Products/monitoring_tool/rules.yml group=accnw name=fix:avg_out:12h >>>>> index=23 msg="Rule evaluation result discarded" err="duplicate sample for >>>>> timestamp" sample="{__name__=\"fix:avg_out:12h\", >>>>> ifAlias=\"FIX_abb_ortakoy_mezarlik_l3vpn.vfrl-@26,330@-||10||\", >>>>> ifDescr=\"Eth-Trunk253.3155\", ifIndex=\"9578\", instance=\"10.85.4.3\", >>>>> job=\"mtx\", sysName=\"1673(3)-PTN3120592_PURSAKLAR_PLAZA\"} => >>>>> 568112.2042330011 @[1670908266553]" >>>>> >>>>> Thanks >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Prometheus Users" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/prometheus-users/0f1c9697-8c58-4895-8917-182a66ed217dn%40googlegroups.com >>>>> <https://groups.google.com/d/msgid/prometheus-users/0f1c9697-8c58-4895-8917-182a66ed217dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>> >>>> >>>> -- >>>> Julius Volz >>>> PromLabs - promlabs.com >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "Prometheus Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-users/6d5070eb-b057-4b6f-9fe6-447cbcfe2c14n%40googlegroups.com > <https://groups.google.com/d/msgid/prometheus-users/6d5070eb-b057-4b6f-9fe6-447cbcfe2c14n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- Julius Volz PromLabs - promlabs.com -- You received this message because you are subscribed to the Google Groups "Prometheus Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-users/CAObpH5yC1LDY1eZKixKN9LB65TcyErb%2B8yfEyRSEqLkrO-1dZQ%40mail.gmail.com.
