[AFMUG] Monitoring for the WIN

[Dennis Burgess]

Just wanted to type up a small experience that I had with our network 
monitoring system.  We have a number of servers that do processing jobs.  These 
are 24 to 40 core Xeon servers that run Windows 10.  We did windows 1 bare 
metal as we wanted the OS to control the power consumption on them as they 
spent quite a bit of time waiting for jobs and not doing anything.

Well, starting last Tuesday, around 11am CST, my Zabbix installation started 
telling me one of our UPSs was in a overload state.  We have overload state set 
to 90%.  This is very odd for us as most of the time we run them under 50% as 
we have a few power switches that allows us to switch between UPSes for 
non-dual power supplied devices.   So I started to review my graphs:

[cid:image001.png@01D74E19.62E82950]

WTF is up wit this, the GREEN line is the current output percentage based on 
the UPS load.  Note I stated it started telling me around 11am on Tuesday that 
we were in a overload state, and I am like no clue what could be causing this.. 
Did someone plug in something?  Like a vacuum or heater and leave it on?  Why 
would anyone be in there?

So I reviewed the security cameras, no one has been in our DC for over 30 days. 
So not that.  Well I looked at our processing servers, and sure enough, one of 
the .EXEs had 45 copies running, each consuming 1.2% of CPU.  Then I looked at 
several other servers, guess what, same thing.  Looked at our DB, the jobs 
completed, but the .exe did not close out.  WTF  I looked at our dev team 
logs and sure enough they updated our processing server .exe right at 11:10am 
on Monday, looking at my logs, guess what started to go up then..

So I killed the .exe that was not closing out, and informed the devs that they 
need to clean up what they are doing.. Screenshot right after I killed the .exe 
on around 10 servers.

[cid:image002.png@01D74E19.62E82950]

So... This spike represents around 2000 watts of power usage that just DROPPED 
... That's 18 amps that we stopped pulling.  Quite a bit!!!  This is what 
Windows 10 power saving and CPU bursting etc., saves us!  Just on one UPS!   
Furthermore, us monitoring and setting triggers allowed us to identify an issue 
that we would normally have never known about.  We adjusted our CPU monitors on 
our processing servers as most of them were around 60-80% used, and adjusted 
them to trigger if the CPU is above 50% for over 10 min, vs 90% for over 5 
minutes.  As the latter never alarmed.



[LTI-Full_175px]
Dennis Burgess

Mikrotik : Trainer, Network Associate, Routing Engineer, Wireless Engineer, 
Traffic Control Engineer, Inter-Networking Engineer, Security Engineer, 
Enterprise Wireless Engineer
Hurricane Electric: IPv6 Sage Level
Cambium: ePMP

Author of "Learn RouterOS- Second Edition"
Link Technologies, Inc -- Mikrotik & WISP Support Services
Office: 314-735-0270  Website: 
http://www.linktechs.net
Create Wireless Coverage's with www.towercoverage.com
Need MikroTik Cloud Management: https://cloud.linktechs.net
How did we do today?
[Gold 
Star][Green
 
Light][Yellow
 
Light][Red
 
Light]

-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] Monitoring for the WIN

[Josh Luthman]

FWIW if you were using Xymon you would have been paged Monday at lunch time
that the load was crazy high on that box.  I would think Zabbix would
notify you that the load changed significantly as well, but I've never used
it.

When something has to chug data I get this:

[image: image.png]

Josh Luthman
24/7 Help Desk: 937-552-2340
Direct: 937-552-2343
1100 Wayne St
Suite 1337
Troy, OH 45373


On Fri, May 21, 2021 at 9:17 AM Dennis Burgess 
wrote:

> Just wanted to type up a small experience that I had with our network
> monitoring system.  We have a number of servers that do processing jobs.
> These are 24 to 40 core Xeon servers that run Windows 10.  We did windows 1
> bare metal as we wanted the OS to control the power consumption on them as
> they spent quite a bit of time waiting for jobs and not doing anything.
>
>
>
> Well, starting last Tuesday, around 11am CST, my Zabbix installation
> started telling me one of our UPSs was in a overload state.  We have
> overload state set to 90%.  This is very odd for us as most of the time we
> run them under 50% as we have a few power switches that allows us to switch
> between UPSes for non-dual power supplied devices.   So I started to review
> my graphs:
>
>
>
>
>
> WTF is up wit this, the GREEN line is the current output percentage based
> on the UPS load.  Note I stated it started telling me around 11am on
> Tuesday that we were in a overload state, and I am like no clue what could
> be causing this.. Did someone plug in something?  Like a vacuum or heater
> and leave it on?  Why would anyone be in there?
>
>
>
> So I reviewed the security cameras, no one has been in our DC for over 30
> days. So not that.  Well I looked at our processing servers, and sure
> enough, one of the .EXEs had 45 copies running, each consuming 1.2% of
> CPU.  Then I looked at several other servers, guess what, same thing…..
> Looked at our DB, the jobs completed, but the .exe did not close out.
> WTF….  I looked at our dev team logs and sure enough they updated our
> processing server .exe right at 11:10am on Monday, looking at my logs,
> guess what started to go up then……
>
>
>
> So I killed the .exe that was not closing out, and informed the devs that
> they need to clean up what they are doing.. Screenshot right after I killed
> the .exe on around 10 servers.
>
>
>
>
>
> So… This spike represents around 2000 watts of power usage that just
> DROPPED … That’s 18 amps that we stopped pulling.  Quite a bit!!!  This is
> what Windows 10 power saving and CPU bursting etc., saves us!  Just on one
> UPS!   Furthermore, us monitoring and setting triggers allowed us to
> identify an issue that we would normally have never known about.  We
> adjusted our CPU monitors on our processing servers as most of them were
> around 60-80% used, and adjusted them to trigger if the CPU is above 50%
> for over 10 min, vs 90% for over 5 minutes.  As the latter never alarmed.
>
>
>
>
>
>
>
> *[image: LTI-Full_175px]*
>
> *Dennis Burgess*
>
>
> * Mikrotik : **Trainer, Network Associate, Routing Engineer, Wireless
> Engineer, Traffic Control Engineer, Inter-Networking Engineer, Security
> Engineer, Enterprise Wireless Engineer*
>
> *Hurricane Electric: **IPv6 Sage Level*
>
> *Cambium: **ePMP*
>
>
>
> Author of "Learn RouterOS- Second Edition”
>
> *Link Technologies, Inc* -- Mikrotik & WISP Support Services
>
> *Office*: 314-735-0270  Website: http://www.linktechs.net
>
> Create Wireless Coverage’s with www.towercoverage.com
>
> Need MikroTik Cloud Management: https://cloud.linktechs.net
>
> *How did we do today?*
>
> [image: Gold Star]
> [image:
> Green Light]
> [image:
> Yellow Light]
> [image:
> Red Light]
> 
>
>
> --
> AF mailing list
> AF@af.afmug.com
> http://af.afmug.com/mailman/listinfo/af_af.afmug.com
>
-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] EBB Enrollment

[dave via AF]

What do we do as an ISP? Do we enroll the customer or do they enroll 
them selves and we submit their acceptance?



On 5/17/21 7:14 AM, Mark Radabaugh wrote:

We have one that we were able to fully qualify and enroll last week.

Of the 5 that applied over the weekend 3 say “Application_Not_Found”, one is 
“Application Pending” and one has multiple fails for 
“Application_not_complete”, “Invalid_Address”, and “State_Fed_Fail”.

Just starting to dig into what all this means though.

Mark


On May 16, 2021, at 3:34 PM, Mark - Myakka Technologies  wrote:

Has  anyone  had  any  luck  enrolling  customers  in  EBB?   I have 3
customers  that  I'm  trying  to enroll.  Al 3 say the went though the
process.  One of them sent me a screen shot saying they were accepted.
Every time I try to enroll them I get

"The  subscriber  has not qualified through the National Verifier yet
or their application has expired. You can qualify them now at 
getemergencybroadband.org"

The  odds  of  all  3 screwing up are low.  Just wondering if there is
something I'm missing.


--

Thanks,
Mark  mailto:m...@mailmt.com

Myakka Technologies, Inc.
www.Myakka.com


--
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com




-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] EBB Enrollment

[James Howard]

We’ve made the decision to require them to do the verification process on the 
website and then give us their information so that we can enroll them.  They 
can’t enroll themselves.  If they do the verification it gives them a number to 
show that they were accepted.  You don’t need to actually enter that number 
when enrolling them but we’re making them give it to us so that we know they 
actually did the verification and we aren’t wasting our time trying to enroll 
them.

From: AF  On Behalf Of dave via AF
Sent: Friday, May 21, 2021 9:02 AM
To: af@af.afmug.com
Cc: dave 
Subject: Re: [AFMUG] EBB Enrollment

What do we do as an ISP? Do we enroll the customer or do they enroll them 
selves and we submit their acceptance?

[cid:image001.jpg@01D74E21.B6ABD200]
On 5/17/21 7:14 AM, Mark Radabaugh wrote:

We have one that we were able to fully qualify and enroll last week.



Of the 5 that applied over the weekend 3 say “Application_Not_Found”, one is 
“Application Pending” and one has multiple fails for 
“Application_not_complete”, “Invalid_Address”, and “State_Fed_Fail”.



Just starting to dig into what all this means though.



Mark



On May 16, 2021, at 3:34 PM, Mark - Myakka Technologies 
 wrote:



Has  anyone  had  any  luck  enrolling  customers  in  EBB?   I have 3

customers  that  I'm  trying  to enroll.  Al 3 say the went though the

process.  One of them sent me a screen shot saying they were accepted.

Every time I try to enroll them I get



"The  subscriber  has not qualified through the National Verifier yet

or their application has expired. You can qualify them now at 
getemergencybroadband.org"



The  odds  of  all  3 screwing up are low.  Just wondering if there is

something I'm missing.





--



Thanks,

Mark  mailto:m...@mailmt.com



Myakka Technologies, Inc.

www.Myakka.com





--

AF mailing list

AF@af.afmug.com

http://af.afmug.com/mailman/listinfo/af_af.afmug.com







Total Control Panel

Login


To: 
ja...@litewire.net

From: af-boun...@af.afmug.com





You received this message because the domain afmug.com is on your allow list.



-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] EBB Enrollment

[Mark Radabaugh]

You very likely want to force them to use the feds website to verify 
eligibility.   You can do it on your own but I would strongly suggest you don’t 
want to do that as it entails a lot more work and liability on your part 
including handling a lot of sensitive personal information like social security 
numbers, tax returns, etc.   

Mark

> On May 21, 2021, at 10:02 AM, dave via AF  wrote:
> 
> What do we do as an ISP? Do we enroll the customer or do they enroll them 
> selves and we submit their acceptance?
> 
> 
> 
> On 5/17/21 7:14 AM, Mark Radabaugh wrote:
>> We have one that we were able to fully qualify and enroll last week.
>> 
>> Of the 5 that applied over the weekend 3 say “Application_Not_Found”, one is 
>> “Application Pending” and one has multiple fails for 
>> “Application_not_complete”, “Invalid_Address”, and “State_Fed_Fail”.
>> 
>> Just starting to dig into what all this means though.
>> 
>> Mark
>> 
>>> On May 16, 2021, at 3:34 PM, Mark - Myakka Technologies  
>>>  wrote:
>>> 
>>> Has  anyone  had  any  luck  enrolling  customers  in  EBB?   I have 3
>>> customers  that  I'm  trying  to enroll.  Al 3 say the went though the
>>> process.  One of them sent me a screen shot saying they were accepted.
>>> Every time I try to enroll them I get
>>> 
>>> "The  subscriber  has not qualified through the National Verifier yet
>>> or their application has expired. You can qualify them now at 
>>> getemergencybroadband.org"
>>> 
>>> The  odds  of  all  3 screwing up are low.  Just wondering if there is
>>> something I'm missing.
>>> 
>>> 
>>> --
>>> 
>>> Thanks,
>>> Mark  mailto:m...@mailmt.com 
>>> 
>>> 
>>> Myakka Technologies, Inc.
>>> www.Myakka.com 
>>> 
>>> 
>>> -- 
>>> AF mailing list
>>> AF@af.afmug.com 
>>> http://af.afmug.com/mailman/listinfo/af_af.afmug.com 
>>> 
>> 
> 
> -- 
> AF mailing list
> AF@af.afmug.com
> http://af.afmug.com/mailman/listinfo/af_af.afmug.com

-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

[AFMUG] SIP External Ringer

[Nate Burke]

I have a request to get a 'Ringer they can hear in the warehouse' When a 
call comes into a ring group.


It looks like SIP ringers start around $300.  Can't I just put in a $30 
ATA and a $30 analog ringer and get the same result?  I'm guessing an 
externally powered ringer would be the best, any recommendations?


--
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] SIP External Ringer

[Steve Jones]

i was just looking at the same thing. was considering using an old r201
with an external analog ringer


On Fri, May 21, 2021 at 12:41 PM Nate Burke  wrote:

> I have a request to get a 'Ringer they can hear in the warehouse' When a
> call comes into a ring group.
>
> It looks like SIP ringers start around $300.  Can't I just put in a $30
> ATA and a $30 analog ringer and get the same result?  I'm guessing an
> externally powered ringer would be the best, any recommendations?
>
> --
> AF mailing list
> AF@af.afmug.com
> http://af.afmug.com/mailman/listinfo/af_af.afmug.com
>
-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] SIP External Ringer

[Adam Moffett]

One of our customers is a machine shop with an external ringer hooked up 
to an ATA.  I have no idea what make/model, but it certainly works.



On 5/21/2021 1:40 PM, Nate Burke wrote:
I have a request to get a 'Ringer they can hear in the warehouse' When 
a call comes into a ring group.


It looks like SIP ringers start around $300.  Can't I just put in a 
$30 ATA and a $30 analog ringer and get the same result?  I'm guessing 
an externally powered ringer would be the best, any recommendations?




--
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] SIP External Ringer

[Daniel White]

With the right parts and pieces, sure.  Sometimes it can be a bit of fun
to get everything to work just right if you are doing applications like
paging. 

CyberData and Algo are the two companies known for SIP enabled stuff. 
An Algo 8180 would probably work great for this but yes it is ~$300.  It
does have a lot of extra features you may or may not use that can
justify the cost.

photograph  
Daniel White
Co-Founder
phone: +1 (702) 470-2770
direct:+1 (702) 470-2766

> Nate Burke 
> May 21, 2021 at 11:40
> I have a request to get a 'Ringer they can hear in the warehouse' When
> a call comes into a ring group.
>
> It looks like SIP ringers start around $300.  Can't I just put in a
> $30 ATA and a $30 analog ringer and get the same result?  I'm guessing
> an externally powered ringer would be the best, any recommendations?
>

-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

[AFMUG] ZOOM issues

[Mark - Myakka Technologies]

Anyone  else  getting  complaints about ZOOM performance over the last
week.   Been getting a higher than usual level of calls about the "Your
Internet  Connection  Is Unstable" message.  These guys are on a fiber
connection.   Our  upstream bandwidth is no where near capacity.  Most
of  these  guys  are  on  our 50/50 mbps plans.  Other real-time stuff
seems to be working, such as VoIP.

I'm just scratching my head on this one.


--

Thanks,
 Mark  mailto:m...@mailmt.com

Myakka Technologies, Inc.
www.Myakka.com


-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] SIP External Ringer

[Steve Jones]

With the r201 in station mode one only would need power, wouldnt need to
run an extension

On Fri, May 21, 2021, 1:37 PM Daniel White  wrote:

> With the right parts and pieces, sure.  Sometimes it can be a bit of fun
> to get everything to work just right if you are doing applications like
> paging.
>
> CyberData and Algo are the two companies known for SIP enabled stuff.  An
> Algo 8180 would probably work great for this but yes it is ~$300.  It does
> have a lot of extra features you may or may not use that can justify the
> cost.
>
> [image: photograph]
> Daniel White
> Co-Founder
> phone: +1 (702) 470-2770
> direct: +1 (702) 470-2766
>
> Nate Burke 
> May 21, 2021 at 11:40
> I have a request to get a 'Ringer they can hear in the warehouse' When a
> call comes into a ring group.
>
> It looks like SIP ringers start around $300.  Can't I just put in a $30
> ATA and a $30 analog ringer and get the same result?  I'm guessing an
> externally powered ringer would be the best, any recommendations?
>
>
> --
> AF mailing list
> AF@af.afmug.com
> http://af.afmug.com/mailman/listinfo/af_af.afmug.com
>
-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] ZOOM issues

[Steve Jones]

Schools are ending, tons of them are streaming movies to students, could be
a saturation issue

On Fri, May 21, 2021, 2:20 PM Mark - Myakka Technologies 
wrote:

> Anyone  else  getting  complaints about ZOOM performance over the last
> week.   Been getting a higher than usual level of calls about the "Your
> Internet  Connection  Is Unstable" message.  These guys are on a fiber
> connection.   Our  upstream bandwidth is no where near capacity.  Most
> of  these  guys  are  on  our 50/50 mbps plans.  Other real-time stuff
> seems to be working, such as VoIP.
>
> I'm just scratching my head on this one.
>
>
> --
>
> Thanks,
>  Mark  mailto:m...@mailmt.com
>
> Myakka Technologies, Inc.
> www.Myakka.com
>
>
> --
> AF mailing list
> AF@af.afmug.com
> http://af.afmug.com/mailman/listinfo/af_af.afmug.com
>
-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

[AFMUG] 48v small pop battery

[TJ Trout]

Anyone have a cheap solution to interface a 48v power supply to a 48v
battery string for small pops?

Ideally you use a DC ups for limiting the charge/discharge but that doubles
the price for even the cheapest option I can find.

What would happen if you just paralleled the load, supply and battery? I'm
not concerned with discharge depth but I'm curious if the power supply may
overload trying to recharge a deeply discharged battery and power the load
simultaneously?
-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] SIP External Ringer

[dave via AF]

+1
the ALGO product is the way to go .. We installed one for a minerals plant.
It has a place for external speaker and a strobe light.


On 5/21/21 1:36 PM, Daniel White wrote:
With the right parts and pieces, sure.  Sometimes it can be a bit of 
fun to get everything to work just right if you are doing applications 
like paging.


CyberData and Algo are the two companies known for SIP enabled stuff.  
An Algo 8180 would probably work great for this but yes it is ~$300.  
It does have a lot of extra features you may or may not use that can 
justify the cost.


photograph  
Daniel White
Co-Founder
phone: +1 (702) 470-2770
direct:+1 (702) 470-2766


Nate Burke 
May 21, 2021 at 11:40
I have a request to get a 'Ringer they can hear in the warehouse' 
When a call comes into a ring group.


It looks like SIP ringers start around $300.  Can't I just put in a 
$30 ATA and a $30 analog ringer and get the same result?  I'm 
guessing an externally powered ringer would be the best, any 
recommendations?







-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] 48v small pop battery

[Mathew Howard]

My solution to that problem is to use a second power supply to charge the
batteries, and put a diode between so that the main power supply can't
charge the batteries. I set the voltage a bit higher on the main power
supply, so that it will take the actual load. That also has the benefit of
giving me redundant power supplies, in case the main one fails.

On Fri, May 21, 2021, 4:32 PM TJ Trout  wrote:

> Anyone have a cheap solution to interface a 48v power supply to a 48v
> battery string for small pops?
>
> Ideally you use a DC ups for limiting the charge/discharge but that
> doubles the price for even the cheapest option I can find.
>
> What would happen if you just paralleled the load, supply and battery? I'm
> not concerned with discharge depth but I'm curious if the power supply may
> overload trying to recharge a deeply discharged battery and power the load
> simultaneously?
> --
> AF mailing list
> AF@af.afmug.com
> http://af.afmug.com/mailman/listinfo/af_af.afmug.com
>
-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com

Re: [AFMUG] 48v small pop battery

[Adam Moffett]

You can get a power supply with adjustable voltage and set it for the 
float voltage on the batteries.


but float voltage is only constant at a constant temperature, so 
maybe not so great in an unconditioned space.



On 5/21/2021 5:30 PM, TJ Trout wrote:
Anyone have a cheap solution to interface a 48v power supply to a 48v 
battery string for small pops?


Ideally you use a DC ups for limiting the charge/discharge but that 
doubles the price for even the cheapest option I can find.


What would happen if you just paralleled the load, supply and battery? 
I'm not concerned with discharge depth but I'm curious if the power 
supply may overload trying to recharge a deeply discharged battery and 
power the load simultaneously?


-- 
AF mailing list
AF@af.afmug.com
http://af.afmug.com/mailman/listinfo/af_af.afmug.com