Re: BANDWIDTH and VONAGE lose FCC rules exemption for STIR/SHAKEN

[Tom Mitchell]

I've seen an uptick, but nothing too dramatic.  Maybe 4-5 junk calls a day
- mostly afternoon.

-- Tom


On Sun, Feb 20, 2022 at 9:57 AM Josh Luthman 
wrote:

> Mine exploded since the requirement date.  Some mornings I get a dozen
> before lunch.
>
> On Fri, Feb 18, 2022 at 2:33 PM Michael Thomas  wrote:
>
>>
>> On 2/17/22 11:58 AM, Sean Donelan wrote:
>> >
>> >
>> https://www.fcc.gov/document/fcc-finds-two-providers-failed-fully-implement-stirshaken-0
>> >
>> >
>> > The Federal Communications Commission today took action to ensure that
>> > voice service providers meet their commitments and obligations to
>> > implement STIR/SHAKEN standards to combat spoofed robocall scams.
>> > Specifically, voice service providers Bandwidth and Vonage lost a
>> > partial exemption from STIR/SHAKEN because they failed to meet
>> > STIR/SHAKEN implementation commitments and have been referred to the
>> > FCC’s Enforcement Bureau for further investigation.
>>
>>
>> So for probably a year or so before the Stir/Shaken mandate came, I have
>> been seeing a lot less phone spam. I don't know if that's typical but it
>> was quite noticeable for me. What that tells me is that providers likely
>> started clamping down on their shady customers well ahead of the mandate
>> which says that regulatory fiat would have been sufficient too. But that
>> hinges on whether my situation is typical though.
>>
>> Mike
>>
>>

Re: New minimum speed for US broadband connections

[Tom Mitchell]

Go Mike!

Seriously, Like Owen, I'm in Evergreen and until recently, my home had very
poor speeds, but at least something.  Today, I have no option other than
Comcast which has jumped to mediocre, and AT&T's DSL.  Seriously. I also
get better service in the Sierra's, but alas, still only one choice.

-- Tom


On Wed, Feb 16, 2022 at 4:52 PM Mike Lyon  wrote:

> If they allow antennas on the roof, we can service them :)
>
> Your house, on the other hand, we already lucked out on that one!
>
> -Mike Lyon
> Ridge Wireless
>
> On Feb 16, 2022, at 16:48, Matthew Petach  wrote:
>
> 
>
>
> On Wed, Feb 16, 2022 at 1:16 PM Josh Luthman 
> wrote:
>
>> I'll once again please ask for specific examples as I continue to see the
>> generic "it isn't in some parts of San Jose".
>>
>
>
> You want a specific example?
>
> Friend of mine asked me to help them get better Internet connectivity a
> few weeks ago.
>
> They live here:
>
> https://www.google.com/maps/place/Meridian+Woods+Condos/@37.3200394,-121.9792261,17.47z/data=!4m5!3m4!1s0x808fca909a8f5605:0x399cdd468d99300c!8m2!3d37.3190694!4d-121.9818295
>
> Just off of I-280 in the heart of San Jose.
>
> I dug and dug, and called different companies.
> The only service they can get there is the 768K DSL service they already
> have with AT&T.
>
> Go ahead.  Try it for yourself.
>
> See what service you can order to those condos.
>
> Heart of Silicon Valley.
>
> Worse connectivity than many rural areas.   :(
>
> Matt
>
>
>

Re: Any sign of supply chain returning to normal?

[Tom Mitchell]

Go virtual.  x86 servers are still 5-8 weeks from our usual suppliers,
although some NICs are 12 weeks and DC Power Supplies are also
52-weeks/'no-idea'.

-- Tom


On Fri, Apr 22, 2022 at 6:21 AM Ryan Wilkins  wrote:

> A company I work for designs a lot of our own hardware and we’ve had a
> number of critical components go EOL suddenly and without warning, such as
> FPGAs, ADCs, clock generators, and SOMs just to name a few.  Just a few
> weeks ago we were informed that a large order of FPGAs was not going to be
> filled at all and the order was cancelled.  Of the parts that aren’t EOL
> (yet), many have 52-week lead times which is just a place holder for “we
> have no idea when we’ll get these” and not an actual delivery estimate.
> Older product lines and lower volume product lines are being cancelled.  We
> had an ADC go EOL because the only factory in Japan making this part burned
> down so not necessarily related to what we think of as supply chain issues,
> but it is of a different sort.
>
> > On Apr 22, 2022, at 8:50 AM, Joe Freeman  wrote:
> >
> > 
> > Basically, anything that uses Broadcom or other commodity silicon is
> currently 55+ weeks out according to most of the vendors I work with.
> Custom Silicon is a bit better or so I'm told, but I've not had to order
> much gear with custom silicon lately, so I've not got a clear read on lead
> times there.
> >
> > I wouldn't be surprised to see some recent gear go End of Sales early
> just because of component shortages and fabs moving to produce the more
> in-demand parts over older less profitable parts.
>

Re: FCC BDC engineer?

[Tom Mitchell]

Reach out to the folks at IP Architechs (https://iparchitechs.com/),
Readitech (https://engr.readitech.com/) or any of the good PE firms and
they can help.

-- Tom


On Tue, Jul 5, 2022 at 8:44 AM KCI Dave Logan via NANOG 
wrote:

> Hi all.  We operate a small regional ISP in Colorado, but no size is too
> small to ignore the FCC, as you all know.
>
> We're really struggling to find the required engineer for the filing, and
> we're small enough that we don't have an officer with engineering
> credentials.
>
> Any pointers in the CO/WY/NE/KS area would be great, on or off list.
>
> I sure hope we're the only org with this problem still, and all the rest
> of you are good to go.
>
> Thanks,
> dave
>
> --
>
> Dave Logan
> Kentec Communications, Inc.
> 970-522-8107
>
>

CUPS in a BNG?

[Tom Mitchell]

Anyone have any thoughts on this CUPS thing?  I have a customer asking, but
it seems the lack of CP resiliency and additional latency between the DP
and CP make this a really dumb idea.  Has anyone tried it?  Does it make
any sense?

Thanks!

Re: CUPS in a BNG?

[Tom Mitchell]

What is it about the architecture that makes it a preferred solution.  I
get that centralizing the user databases makes sense, but why the control
plane.  What benefit does that have?

-- Tom


On Wed, Mar 22, 2023 at 2:17 PM  wrote:

> The CUPS makes a lot of sense for this application. Latency is dependent
> on the design, and equipment used. I’ve seen/done several designs for this
> using two different vendors equipment and two different BNG software stacks.
>
> When I do a design for BNG from scratch, this is how I do it now. :)
>
> As always… YMMV.
>
> - Brian
>
> On Mar 22, 2023, at 4:02 PM, Tom Mitchell 
> wrote:
>
> Anyone have any thoughts on this CUPS thing?  I have a customer asking,
> but it seems the lack of CP resiliency and additional latency between the
> DP and CP make this a really dumb idea.  Has anyone tried it?  Does it make
> any sense?
>
> Thanks!
>
>
>

Re: CUPS in a BNG?

[Tom Mitchell]

OK - That makes sense.  For scaling a CP, it only about redundancy,
correct, but with the DP it's really about scaling up and out. But still, a
CP is no longer on the bus with the DP, nor on the network.  It's on the
WAN/Internet, and latencies are orders of magnitude greater.  Is anybody
doing this and are those latencies acceptable?



On Wed, Mar 22, 2023 at 2:59 PM Joel Halpern  wrote:

> With a reasonable design, it separates the scale issues of the control
> plane from the scale issues of the data plane.  And since the relationship
> between those two scale factors is different for different deployments, it
> allows you as an operator to build for your needs.  It also, with suitable
> designs separates the failure modes.
>
> Whether either of those applies in your case probably depends upon your
> needs and what vendors you find useful.
>
> Yours,
>
> Joel
> On 3/22/2023 5:53 PM, Tom Mitchell wrote:
>
> What is it about the architecture that makes it a preferred solution.  I
> get that centralizing the user databases makes sense, but why the control
> plane.  What benefit does that have?
>
> -- Tom
>
>
> On Wed, Mar 22, 2023 at 2:17 PM  wrote:
>
>> The CUPS makes a lot of sense for this application. Latency is dependent
>> on the design, and equipment used. I’ve seen/done several designs for this
>> using two different vendors equipment and two different BNG software
>> stacks.
>>
>> When I do a design for BNG from scratch, this is how I do it now. :)
>>
>> As always… YMMV.
>>
>> - Brian
>>
>> On Mar 22, 2023, at 4:02 PM, Tom Mitchell 
>> wrote:
>>
>> Anyone have any thoughts on this CUPS thing?  I have a customer asking,
>> but it seems the lack of CP resiliency and additional latency between the
>> DP and CP make this a really dumb idea.  Has anyone tried it?  Does it make
>> any sense?
>>
>> Thanks!
>>
>>
>>

Re: 10G CPE w/VXLAN - vendors?

[Tom Mitchell]

Lanner NCA-1516

-- Tom


On Fri, Jun 16, 2023 at 9:15 AM Michel Blais 
wrote:

> Pretty sure Ufispace S9502-12SM + IPInfusion OcNOS would work. VxLAN is
> supported in the IP Base licence. CRS license must be avoided for VxLAN.
> Look at the OcNOS feature matrix to make sure.
>
> Le mer. 14 juin 2023 à 14:51, Adam Thompson  a
> écrit :
>
>> Hello, all.
>>
>> I’m having difficulty finding vendors, never mind products, that fit my
>> need.
>>
>>
>>
>> We have a small but growing number of L2 (bridged) customers that have
>> diverse fiber paths available, and, naturally, want to make use of them.
>>
>> We have a solution for this: we extend the edge of our EVPN VXLAN fabric
>> right to the customer premise.  The customer-prem device needs 4x10G SFP+
>> cages (2 redundant paths, plus LAG to customer), and the switches we
>> currently use, Arista 7020Rs, are quite expensive if I’m deploying one one
>> per customer.  (Nice switches, but overkill here – I don’t need 40/100G,
>> and I don’t need 24 SFP+ ports.  And they still take forever to ship.)
>>
>>
>>
>> We use RFC7438 §6.3 “vlan-aware-bundle” mode, not §6.1 “vlan-based” mode,
>> which limits our choices somewhat.  I might be willing to entertain
>> spinning up a separate VXLAN mesh using RFC7438 §6.1 (“vlan-based”) and
>> static VTEPs if it saves me a lot of pain.
>>
>>
>>
>> However, I’m having trouble finding small & cheap*er* 1U (or even
>> desktop/wallmount) devices that have 4 SFP+ cages, and can do VXLAN, in the
>> first place.
>>
>> Who even makes CPE gear with SFP+ ports?  (Other than Mikrotik
>> CRS309-1G-8S+IN / CRS317-1G-16S+RM, which are nice, but our policy requires
>> vendor support contracts, so… no-go.)
>>
>>
>>
>> Vendors?  Model#s, if you happen to know any?
>>
>>
>>
>> Reply here or privately, whatever floats your boat – any pointers
>> appreciated!
>>
>>
>>
>> *Adam Thompson*
>>
>> Consultant, Infrastructure Services
>>
>> [image: [MERLIN logo]]
>>
>> 100 - 135 Innovation Drive
>>
>> Winnipeg, MB R3T 6A8
>>
>> (204) 977-6824 or 1-800-430-6404 (MB only)
>>
>> https://www.merlin.mb.ca
>>
>> Chat with me on Teams
>> 
>>
>>
>>
>

Re: CPE/NID options

[Tom Mitchell]

I don't know about specific SKUs, but IP Infusion make a very popular set
of L2 switches.


On Wed, Nov 22, 2023 at 8:42 PM Ross Tajvar  wrote:

> I'm evaluating CPEs for one of my clients, a regional ISP. Currently,
> we're terminating the customer's service (L3) on our upstream equipment and
> extending it over our own fiber to the customer's premise, where it lands
> in a Juniper EX2200 or EX2300.
>
> At a previous job, I used Accedian's ANTs on the customer prem side. I
> like the ANT because it has a small footprint with only 2 ports, it's
> passively cooled, it's very simple to operate, it's controlled centrally,
> etc. Unfortunately, when I reached out to Accedian, they insisted that the
> controller (which is required) started at $30k, which is a non-starter for
> us.
>
> I'm not aware of any other products like this. Does anyone have a
> recommendation for a simple L2* device to deploy to customer premises? Not
> necessarily the exact same thing, but something similarly-featured would be
> ideal.
>
> *I'm not sure if the ANT is exactly "layer 2", but I don't know what else
> to call it.
>

Re: sigs wanted for a response to the fcc's NOI for faster broadband speeds

[Tom Mitchell]

Not sure we need the FCC telling us how to build products or run networks.
Seat belts are life-or-death, but bufferbloat is rarely fatal ;-)  Let it
be a point of differentiation.

-- Tom


On Thu, Nov 30, 2023 at 4:56 PM Dave Taht  wrote:

> Over here:
>
>
> https://docs.google.com/document/d/19ADByjakzQXCj9Re_pUvrb5Qe5OK-QmhlYRLMBY4vH4/edit
>
> Us bufferbloat folk have been putting together a response to the FCC's
> NOI (notice of inquiry) asking for feedback as to increasing the
> broadband speeds beyond 100/20 Mbit.
>
> "Calls for further bandwidth increases are analogous to calling for
> cars to have top speeds of 100, 200, or 500 miles per hour. Without
> calling also for better airbags, bumpers, brakes, or steering wheels,
> (or roads designed to minimize travel delay), these initiatives will
> fail (and are failing) to meet the needs of present and future users
> of the internet."
>
> Comments (and cites) welcomed also! The text is still somewhat in flux...
>
>
> --
> :( My old R&D campus is up for sale: https://tinyurl.com/yurtlab
> Dave Täht CSO, LibreQos
>

Re: CGNAT growing pains

[Tom Mitchell]

Hi Jon,

Are you dual stack?  v6 would solve some of these issues?



On Tue, Oct 8, 2024 at 12:20 PM Jon Lewis  wrote:

> We started rolling out CGNAT about 6 months ago.  It was smooth sailing
> for the first few months, but we eventually did run into a number of
> issues.
>
> Our customer base is primarily FTTH with "dynamic" IP assignment via DHCP.
> Since connections are always-on, customer ONTs/routers get an IP assigned,
> and then when the lease is renewed, they request a new lease for the
> existing IP, and, in general, that request is granted.  This gives
> customers the mistaken impression they have a static IP.  So, my
> impression, from working with some customers who've needed to be moved
> from CGNAT back to public IP is that customers who are doing
> port-forwarding don't even bother with dynamic DNS.  They just know they
> can connect to their IP as they've never seen it change.  We do offer/sell
> static IP, but pre-CGNAT, it was strictly for business customers.  i.e.
> A residential customer could only get static IP service by converting
> their account to a business account. That may change in the near future.
>
> One issue we didn't foresee has been IP Geo issues.  i.e.  We all knew
> that streaming services like Netflix use IP Geo to determine what content
> should be made available, but that's, AFAIK, limited by country or region.
> What we didn't anticipate is services like Hulu Live TV doing IP Geo down
> to the city level to determine which local channels are a subscriber's
> local channels.  We're using Juniper MX gear and SPC3 cards for our CGNAT
> routers, each one having a single large external pool.  Since we serve
> most of FL, one external pool can't IP Geo correctly for customers as far
> apart as Miami and Jacksonville hitting the same CGNAT router.  We don't
> currently have an acceptable solution to this other than moving impacted
> customers off CGNAT.
>
> One of the great unknowns (at least for us) with CGNAT was what our PBA
> settings should be.  i.e.  How large each port-block should be, and how
> many port-blocks to allow per customer.  We started with 256x4.  It seemed
> to work.  We eventually noticed that we were logging port-block exceeded
> errors.  This is one aspect where Juniper's CGNAT support is lacking.
> There's a counter for these errors, and it's available via SNMP, but
> there's no way to attribute the errors to subscriber IPs.  We're polling
> the mib and graphing it, so we know it's a continuing issue and can see
> when it's incrementing faster/slower, but Junos provides no means for
> determining if "PBEs" are all being caused by a single customer, a handful
> of customers, etc.  We have a JTAC case open on this.  As a quick &
> hopeful fix, we both increased the port-block size and block limit.  That
> helped, but didn't stop the errors.  It also cut our CGNAT ratio by more
> than half (64:1 -> 28:1), if we stay at this ratio, we'll need much larger
> external pools than originally anticipated.  Tuning these settings is kind
> of painful as JTAC strongly recommends bouncing the CGNAT service anytime
> CGNAT related config changes are made.  This means briefly breaking
> Internet access for all CGNAT'd customers.  For the PBEs, JTAC's
> suggestions so far have been to shorten some of the timeouts in the config
> and to keep doing what we're doing, which is a cron job that essentially
> does a "show services nat source port-block", parses the output looking
> for subscriber IPs that have used up the ports in several of their
> port-blocks, then does a "show services sessions source-prefix ..." and
> logs all of this.  This at least gives us snapshots of "who's a heavy user
> right now" and lets us look at how they were using all their ports.  i.e.
> was it bittorent, are they compromised and scanning the internet for more
> systems to compromise, is it legit looking traffic - just lots of it,
> etc.?
>
> The latest CGNAT issue is a customer with a Palo Alto Networks firewall
> connected to our network and several of their employees are our FTTH
> customers.  On their PANW firewall, they're doing IP Geo based filtering,
> limiting access to internal servers to "US IPs".  Since we only CGNAT
> traffic to the external Internet, their on-net employees hit the firewall
> from their 100.64/10 IPs and get blocked.  I suggested they whitelist
> 100.64/10, saying we block traffic from 100.64/10 from entering our
> network via peering and transit, so they can be assured anything from
> 100.64/10 came from inside our network / our customers.  They say the
> firewall won't let them whitelist 100.64.0.0/10, giving an error that
> it's
> invalid IP space.
>
> I know we're not the first to implement CGNAT, so I'm curious if others
> have run into these sorts of issues, or others we haven't run into yet,
> and if so, how you solved them.
>
>
> --
>   Jon Lewis, MCP :)  |  I route
>   

Re: CGNAT growing pains

[Tom Mitchell]

https://www.google.com/intl/en/ipv6/statistics.html



On Tue, Oct 8, 2024 at 1:19 PM Jon Lewis  wrote:

> I'm not so sure about that.  Our customers are all offered dual-stack
> (DHCPv6, DHCPv6-PD).  Do any of the common streaming services support v6
> yet?  Last I checked, Hulu did not.
>
> On Tue, 8 Oct 2024, Michael Thomas wrote:
>
> > Hi Jon,
> >
> > So is this easier than what the mobile carriers are doing -- 464xlat,
> isn't
> > it? Probably a sizeable portion of the traffic would be running native
> v6,
> > right? Obviously it wouldn't run into these sorts of problems.
> >
> > Mike
> >
> > On 10/8/24 12:19 PM, Jon Lewis wrote:
> >>  We started rolling out CGNAT about 6 months ago.  It was smooth sailing
> >>  for the first few months, but we eventually did run into a number of
> >>  issues.
>
> --
>   Jon Lewis, MCP :)  |  I route
>   Blue Stream Fiber, Sr. Neteng  |  therefore you are
> _ http://www.lewis.org/~jlewis/pgp for PGP public key_
>

Re: New home builders without wires

[Tom Mitchell]

Beyond that, my home (c. 1996) has RJ-45 stapled to the studs every X feet,
jacks in every room, and super-fat coax similarly fastened to studs in 3
rooms.  Of course, none of this is in use.  How many times have I wished
they used conduit.


On Tue, Dec 3, 2024 at 10:43 AM Colin Stanners (lists) <
colin-li...@highspeedcrow.ca> wrote:

> It's easy and relatively cost-effective to make a new home pretty
> future-proof for connectivity by running conduit (of sufficient size,
> without tight bends) from the telecom area to the outlet box(es) in each
> room. For today run a coax and one or two cat6A; then whatever system
> appears in the future can quickly replace those in the conduit. Considering
> the importance of telecommunication/entertainment it's a surprise that very
> few new homes seem to take that option, but I guess it's not "trendy"
> enough.
>
> For the below example, it seems like an extreme example of cost-cutting,
> along with believing that "wireless is magic". When the real-world concerns
> about coverage and capacity appears, and the residents have 5 smart TVs
> competing to stream video on Wi-Fi along with game consoles downloading
> 100GB games (all of which should have been wired-in), is when they realize
> the difficulties of not planning the network and layout.
>
> By that time the builder will be long gone with the money...
>
>
> -Original Message-
> From: NANOG  On
> Behalf
> Of Sean Donelan
> Sent: December 3, 2024 10:53 AM
> To: nanog@nanog.org
> Subject: New home builders without wires
>
> As some may remember from earlier this year, my friend was buying a new
> "semi-custom" home.  "Semi-custom" is a marketing term, meaning you get to
> choose (pay more) pre-determined builder options. It is not custom
> designed.
>
> The home builder was not installing any wired broadband utilities in the
> new
> neighborhood.  No cable coax, no telephone DSL, no fiber optic. The only
> option was wireless, with a special deal with a specific 5G wireless
> cellular provider.
>
> Originally, the builder's sales agent (i.e. the people working in the model
> home selling houses) said new homes didn't need (and would not have) a
> wired
> "demarc" location and no ethernet or coax outlets. Not my house, but I was
> surprised when I heard that. I like wired connections when possible for any
> fixed devices, and WiFi only for mobile devices.
>
> I visited his new house over the Thanksgiving Holiday.
>
> The sales agent was partially wrong and partially correct. Never believe
> the
> sales agent spiel.
>
> The built house came with exactly FOUR wired ethernet outlets in the living
> room and each bedroom/office (x2 Cat6 jacks each outlet). But no wired
> DEMARC, no coax outlets, and no wired broadband utilities in the neighhood.
> The wired ethernet jacks were needed because the wireless 5G base station
> ended up in an upstairs bedroom window for signal strength reasons. The
> in-house wired ethernet was needed for a WiFi extender in the living room.
>
> I wouldn't be happy, but it seems to work for his family. The 5G deal was
> cheaper than what he was paying at his old house.
>
> According to the real estate realtor, not the builder's sales agent,
> broadband is now in the top three things home buyers want to know. Some
> states require the realtor MLS to disclose broadband access in the home
> listings. Broadband access disclosure not required in this state.
>
>