Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/25/20 11:04 AM, Mark Tinka wrote:



On 12/25/20 20:10, Hunter Fuller via NANOG wrote:



It would meet some customers’ needs because multiple people could use 
1G of service at a time. I think it is interesting to distinguish 
“>1G CPE” from “true 10G CPE” and I suspect many / most customers are 
looking for the former.


Large upstream capacity has always been about aggregation of the 
downstream.





Can I ask a really dumb question? Consider it an xmas present. I know 
this sounds like "nobody needs more than 640k", but how can household 
possibly need a gig let alone 10g? I'm still on 25mbs DSL, have cut the 
cord so all tv, etc is over the net. If I really cared and wanted 4k I 
could probably upgrade to a 50mbs service and be fine. Admittedly it's 
just the two of us here, but throw in a couple of kids and I still don't 
see how ~100mbs isn't sufficient let alone 1 or 10G. Am I missing 
something really stupid?


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/25/20 11:39 AM, Cory Sell wrote:
I saturate my 1G connection most during game downloads, file 
downloads/uploads, full backup uploads, etc.


I also self-host a lot of services for personal use and having that 
peak speed is really nice when you need it. It also had no traffic 
limit per month which is my biggest complaint about the lower tier 
services and also a huge complaint I have with regards to the 
direction that residential services are moving towards.



Obviously for downloads it's nice, but how often is that happening? A 
time or two a month max? It seems sort of strange the providers would 
build out infrastructure for such a niche activity.


Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/25/20 11:34 AM, Niels Bakker wrote:

* mark.ti...@seacom.com (Mark Tinka) [Fri 25 Dec 2020, 19:11 CET]:
I have a mate up the road who just paid for a 1Gbps FTTH service 
because it was the same price as a 100Mbps one. He generally lives 
between 900Kbps and 20Mbps.


Gigabit-level FTTH services for the home, I feel, have always been 
about marketing ploys from providers, because they know there is no 
practical way users can ever hit those figures from their homes.

[...]

Gigabit speeds are about bursting.  Foreground activities like gaming, 
making online reservations, streaming won't take more than that, but 
anything faster is really nice to have when you're waiting for the odd 
software download to finish. (You may have noticed that they've been 
increasing in size this year.)


Wouldn't cpe that implements proper queuing disciplines be a lot simpler 
and cheaper? I got bit by that once when a friend was downloading a game 
and it. I flashed a router with openwrt and fiddled with their queuing 
nobs and everything was golden.


Mark is probably right though: it's just marketing. Who would have 
believed that bandwidth would just become a marketing ploy.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/25/20 12:40 PM, Chris Adams wrote:


The other aspect of it is that we're doing these downloads while
continuing to play other games and chat (both things sensitive to
latency).  Some have family/roommates in the home, so they may be
streaming audio and/or video at the same time.  Do we fill up a gigabit?
No, probably not... but we'd notice if we had a lot less.


But using the right queuing disciplines it a lot cheaper than the brute 
force and ignorance of just upping the bandwidth, right?


It seems really surprising after almost a decade of discovery of 
bufferbloat that most CPE are still doing tail drops.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/25/20 12:53 PM, Chris Adams wrote:

Once upon a time, Michael Thomas  said:

On 12/25/20 12:40 PM, Chris Adams wrote:

The other aspect of it is that we're doing these downloads while
continuing to play other games and chat (both things sensitive to
latency).  Some have family/roommates in the home, so they may be
streaming audio and/or video at the same time.  Do we fill up a gigabit?
No, probably not... but we'd notice if we had a lot less.

But using the right queuing disciplines it a lot cheaper than the
brute force and ignorance of just upping the bandwidth, right?

Queueing doesn't get me my next game in time to play it tonight.  I've
always seen general queueing as a work-around for "not enough bandwidth
and can't add more"... but when more is available, why not just use
more?

I'm fine with "free stuff". But it seems we've hit saturation on a 
number of front like camera and screen pixels, ghz of cpu, TB's of disk, 
Gb's of netio for residential stuff.


My provider on the other (Volcano Internet) doesn't seem to have got 
this memo though. They are building out fiber and the rate sheet is the 
same as for DSL. I mean, wtf? Why would I want the probable expense of 
getting it from the curb (assumedly) to my home if it's for the same 
price? Even if it's ftth at their expense, it seems rather pointless.


I mean, i understand the arm's race, but now it seems to be an arms race 
for its own sake.


Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/25/20 1:22 PM, Niels Bakker wrote:


Wouldn't cpe that implements proper queuing disciplines be a lot 
simpler and cheaper? I got bit by that once when a friend was 
downloading a game and it. I flashed a router with openwrt and 
fiddled with their queuing nobs and everything was golden.


Let's take an example from earlier this year when Activision shipped a 
180GB update to Call of Duty: Modern Warfare when they introduced the 
War Zone BR game mode update.


Download times:-

180GB at 100 Mbps: 4 hours
180GB at 1000 Mbps: 23 minutes

How will proper queuing disciplines possibly help here?



The queuing disciplines allow you to not completely hog the bandwidth so 
that other people can use the net too. Tail drop seems to rule the roost 
to this day with CPE so it must be a real joy when you're downloading them.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/25/20 1:25 PM, John Levine wrote:

In article  you write:

I'm fine with "free stuff". But it seems we've hit saturation on a
number of front like camera and screen pixels, ghz of cpu, TB's of disk,
Gb's of netio for residential stuff.

My provider on the other (Volcano Internet) doesn't seem to have got
this memo though. They are building out fiber and the rate sheet is the
same as for DSL. I mean, wtf?

How fast is your DSL?  It looks like your provider's DSL tops out at 50/5, which
I suspect is not available everywhere, while fiber starts at 25/25 and goes to 
100/100.

I rather like the 100/100 symmetrical bandwidth on my fiber. I can
assure you that 100/100 feels noticably faster than 25/5 even though
nothing here would use even 25Mb sustained.


They max out at 50, which i might be able to get since I think the 
pedestal is about 1/2 mile away. When I was with Sonic they have that 
Fusion product but I think I could only get 50 because I was about 9000' 
from the CO in SF.


I'd definitely appreciate symmetric, or at least better in upstream. 
Obviously zoom and all of that has made a lie of us not needing 
upstream. It would make cloud based "filesystems" more feasible too.


But the larger point is why bother going to all of that effort if you're 
just going roll it out with low bandwidth? I mean, 100Mbps isn't even 
competitive with cable these days. But they're a somewhat crazy amalgam. 
They have POTS everywhere, cable tv everywhere, cable IP in some areas 
and DSL in others. I wish I knew somebody there to talk to this about 
because it's really odd.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/25/20 2:32 PM, John Levine wrote:

In article <3b0bc95b-c741-7561-1692-75fac74d5...@mtcc.com> you write:

I'd definitely appreciate symmetric, or at least better in upstream.
Obviously zoom and all of that has made a lie of us not needing
upstream. It would make cloud based "filesystems" more feasible too.

But the larger point is why bother going to all of that effort if you're
just going roll it out with low bandwidth? I mean, 100Mbps isn't even
competitive with cable these days. But they're a somewhat crazy amalgam.
They have POTS everywhere, cable tv everywhere, cable IP in some areas
and DSL in others. I wish I knew somebody there to talk to this about
because it's really odd.

I agree it is odd to make 100/100 the top speed. The fiber service I
have from my local non-Bell telco offers 100/100, 500/500, and
1000/1000. FiOS where you can get it goes to 940/880.

The obvious guess is that their upstream bandwidth is
underprovisioned, or maybe they figure 100/100 is all they need to
compete in that particular market.


What's weirder is that it's most likely not going to allow them to 
retire their copper plant since they are a phone company and i'm fairly 
certain that regulations won't allow them to say "get a battery for this 
phone dongle". Given PG&E's antics, this is no small thing. I assume it 
would allow them to retire their cable plant eventually, but then they 
become yet another over the top provider without adding much if any 
value. But they are an odd and very old family run company, so who knows 
what's going on in the C-Suite.


Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/26/20 8:00 AM, Valdis Klētnieks wrote:


Anybody got a feel for what percent of the third-party gear currently sold to
consumers has sane bufferbloat support in 2020, when we've *known* that
de-bufferbloated gear is a viable differentiatior if marketed right (consider 
the
percent of families that have at least one gamer who cares)?

I don't know percentages, but just trying to find cpe that support it in 
their specs is depressingly small. considering that they're all using 
linux and queuing discipline software is ages old, i really don't get 
what the problem is. it's like they're being deliberately obtuse. given 
all of the zoom'ing happening now you think that somebody would hit them 
with the clue-bat that this is a marketing opportunity.


Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/26/20 9:50 AM, Mel Beckman wrote:

i really don't get what the problem is. it's like they're being deliberately 
obtuse.

Michael,

If vendors saw a 10GbE CPE market, they would serve it. Obviously they don’t 
see a market. Why don’t people insisting vendors build their hobby horse see 
that? It’s like they’re being deliberately obtuse :)

The thing is that the pandemic has changed the game on the ground: there 
is an actual feature differentiator to be had. But having dealt with the 
Linksys folks in the past I don't put out much hope that they'll take 
advantage of it. The software development side was a vast black hole 
where time stands still. It seems the entire industry is like that.


Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/26/20 10:00 AM, Tony Wicks wrote:

Actually the equipment vendor's build in this sort of situation is normally 
directly related to the availability of affordable chipsets from the likes of 
Broadcom. For example the chipset in my XGSPON router is a BCM6858. No vendor 
is going to spend money to produce a CPE that no one will buy. Once the likes 
of Broadcom produce an affordable solution then all the main vendors will roll 
out CPE in short order.


Do they have no control of the linux kernel? This is purely OS kernel 
work and has nothing to do with underlying hardware.


Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/26/20 10:09 AM, Valdis Klētnieks wrote:

On Sat, 26 Dec 2020 17:50:28 +, Mel Beckman said:

If vendors saw a 10GbE CPE market, they would serve it. Obviously they don’t
see a market. Why don’t people insisting vendors build their hobby horse see
that? It’s like they’re being deliberately obtuse :)

The number of people that want a router that does 10GbE is vastly
outnumbered by the number of people that want a router that
makes their Zoom sessions not suck.

Admittedly, many of them don't realize they want that router, mostly
because most of them don't realize it's not difficult at all to build one
that does that.  But that's why companies have an advertising and marketing
team. :)


The marketing writes itself:

"Do you have to kick your kids of the network for company Zoom calls? 
You need this brand spanking new router!"


I've been trying to explain to friends that are now saddled with video 
calls all the time what the problem is, but it's really hard to point 
and say "buy this router". There are a few out there that feature it, 
but they're about $200 which is pretty spendy. Considering that this is 
just a OS module, your basic $50 router should be able to support it 
without any problem too.


Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/26/20 11:49 AM, Mel Beckman wrote:
The thing is that the pandemic has changed the game on the ground: 
there is an actual feature differentiator to be had. But having dealt 
with the Linksys folks in the past I don't put out much hope that 
they'll take advantage of it. The software development side was a 
vast black hole where time stands still. It seems the entire industry 
is like that.


Michael,

Even 100 Mbps Internet is fine for Zoom, as long as the uplink speed 
is at least 10 Mbps. The average zoom session requires 2 Mbps up and 
down, and even for the lavish six-screen executive sessions, 6 Mbps is 
plenty good. So arguing that 10 GbE is necessary because “pandemic has 
changed the game on the ground” is silly.


https://support.zoom.us/hc/en-us/articles/204003179-System-requirements-for-Zoom-Rooms#h_b48c2bfd-7da0-4290-aae8-784270d3ab3f 



So, sorry, 10 GbE is a hobbyists fantasy, not a marketable product. If 
hobbyists want 10GbE, let them pay for it like the rest of us, and let 
them play CoD from inside  freezing data center :)


I'm not saying anything about 10G, other than my initial query as to 
whether any residence could possibly need that much bandwidth. But 
buffer bloat is a problem with a lot of us still stuck on DSL with no 
prospect of anything better. It's not the bandwidth per se, it is how 
the bandwidth is consumed at home. Better queuing disciplines than tail 
drop with a gigantic queue could help zoom meetings a lot where 
bandwidth is more constrained. And regardless of bandwidth, huge queues 
are not good for real time traffic for anything. You'd think that gamers 
would be acutely aware of this and create a market for routers that 
cater to their hunger for less latency.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/26/20 12:44 PM, John Levine wrote:


In the 25 years since I've lived here the power has never been out as
long as a day so I think a four day battery will give me pretty good
reliability. I know my fiber is a straight shot to the CO since I'm
only four blocks away but as far as I can tell, unlike the HFC cable
plant next to it on the poles, their fiber system doesn't use any
powered repeaters.

Here in California the new reality is that multi-day outages are now 
common. The first few planned outages were 3-4 days, so that would be on 
the edge, especially if it's for gabby granny on the phone for 
hours.This all depends on the weather, and for snow related outages they 
can go on for days. We have a generator because of this, but everybody 
getting a generator in the middle of the Berkeley Hills would be 
something of its own horror show, but it will probably come down to that.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/26/20 1:13 PM, Valdis Klētnieks wrote:

On Sat, 26 Dec 2020 12:58:42 -0800, Michael Thomas said:

can go on for days. We have a generator because of this, but everybody
getting a generator in the middle of the Berkeley Hills would be
something of its own horror show, but it will probably come down to that.

Egads.

Especially if a lot of those generators are just bought at Home Depot and
hooked up to the house wiring without a proper cutover switch for the mains.




Yeah, it burned somebody's house to a crisp here last year around here. 
It certainly makes the case why leaving professionals in charge of power 
issues is the better idea. although with pg&e it's a tough call, my 
telco not so much.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/26/20 3:28 PM, Mike Hammett wrote:
Use a router with FQ_CODEL and be amazed at how much you can get onto 
a pipe without any perceptible difference in the experience.




I did that, after a meltdown and yes it made a huge difference. I don't 
understand why CPE don't implement it by default.



Mike





-
Mike Hammett
Intelligent Computing Solutions <http://www.ics-il.com/>
<https://www.facebook.com/ICSIL><https://plus.google.com/+IntelligentComputingSolutionsDeKalb><https://www.linkedin.com/company/intelligent-computing-solutions><https://twitter.com/ICSIL>
Midwest Internet Exchange <http://www.midwest-ix.com/>
<https://www.facebook.com/mdwestix><https://www.linkedin.com/company/midwest-internet-exchange><https://twitter.com/mdwestix>
The Brothers WISP <http://www.thebrotherswisp.com/>
<https://www.facebook.com/thebrotherswisp><https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg>
--------
*From: *"Michael Thomas" 
*To: *nanog@nanog.org
*Sent: *Friday, December 25, 2020 1:27:39 PM
*Subject: *Re: [External] Re: 10g residential CPE


On 12/25/20 11:04 AM, Mark Tinka wrote:



On 12/25/20 20:10, Hunter Fuller via NANOG wrote:


It would meet some customers’ needs because multiple people
could use 1G of service at a time. I think it is interesting
to distinguish “>1G CPE” from “true 10G CPE” and I suspect
many / most customers are looking for the former.


Large upstream capacity has always been about aggregation of the
downstream.



Can I ask a really dumb question? Consider it an xmas present. I know 
this sounds like "nobody needs more than 640k", but how can household 
possibly need a gig let alone 10g? I'm still on 25mbs DSL, have cut 
the cord so all tv, etc is over the net. If I really cared and wanted 
4k I could probably upgrade to a 50mbs service and be fine. Admittedly 
it's just the two of us here, but throw in a couple of kids and I 
still don't see how ~100mbs isn't sufficient let alone 1 or 10G. Am I 
missing something really stupid?


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/27/20 2:26 AM, Mark Tinka wrote:



On 12/26/20 23:57, Michael Thomas wrote:



Yeah, it burned somebody's house to a crisp here last year around 
here. It certainly makes the case why leaving professionals in charge 
of power issues is the better idea. although with pg&e it's a tough 
call, my telco not so much.


I considered a generator at some point, for home back up.

In the end, and for various reasons, I settled on renewables.

I'm just not sure where all that Li-Ion will go after 15 - 20 years of 
use, though...


One European manufacturer (the one whose battery I bought) says that 
as of now, they can only recycle 20% of each battery they sell. To me, 
that sounds like just the metal case enclosure, and the plastic facia.


Ah well, maybe disposal tech. for Li-Ion storage will have improved by 
2040.




We have both, and are going to get a battery. But the battery would 
probably only be good for about a day which is not enough, especially 
with these planned shutoffs because they have to inspect their wire 
plant in daylight. There has to be a better technical solution for this 
beyond just burying the wires. A properly trained AI could probably 
figure out what's naught and nice.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/27/20 9:38 AM, Mark Tinka wrote:



On 12/27/20 18:14, Michael Thomas wrote:

We have both, and are going to get a battery. But the battery would 
probably only be good for about a day which is not enough, especially 
with these planned shutoffs because they have to inspect their wire 
plant in daylight.


If you can add some solar panels to that, you would be in a better 
position to prolong the battery's utility.


I'd say dump the generator, and invest that money in solar panels, 
rather. Batteries are way more costly than panels, and if you can have 
both, you're going to be better off in the long run.


We can't get enough solar panels on the roof to charge a battery big 
enough to handle a multi-day outage, and the battery as quoted is only 
charged from the panels, not from the mains. It's easy enough to get a 
transfer switch though for the battery subpanel to hook the generator up 
to. If I really wanted to get fancy, I could supply the generator from 
our house propane tank, but it's not that hard to just use the normal 5 
gallon type tanks.


Mike, it's sunday so i guess it's ok to be off topic :)

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/27/20 10:00 AM, Valdis Klētnieks wrote:

On Sun, 27 Dec 2020 17:57:17 +0100, Baldur Norddahl said:


Here in the civilised world we bury the wires ;-)

Even the long-haul 765kv and up connections across the power grid?

In the US, they're out on towers for a reason - you can fly along them
in a helicopter and easily spot parts of cable that are degrading and need
repair because they glow brighter on an infrared scope...

(Plus, as Hurricane Sandy taught Manhattan, buried wires have their
own rather nasty failure modes)


Right and here in California, it was precisely those lines that 
incinerated Paradise. The problem with PG&E is that they couldn't be 
bothered to maintain anything since it got in the way of cushy estaff 
salaries and investor dividends. The tower that caused Paradise was a 
century old.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/27/20 10:26 AM, Baldur Norddahl wrote:
All of the 400V and 10 kV is buried. That means no wires along 
streets, anywhere.


The long haul transmission network consists mostly of 150 kV and 400 
kV lines. That has been partly buried, especially near and in cities. 
There was a project to have it all buried but was abandoned halfway 
due to cost.


But then it is all fully redundant, so they will just power it down if 
it needs maintenance. My company is digging for FTTH and in the few 
cases we need to cross one of these bad guys, they will shut it for us 
while we are working. Nobody looses power of course.


The 10 kV network is redundant too. We managed to hit those a few 
times.  That will cause a power interruption for 10 to 20 minutes 
until they reroute the power. I believe mostly for safety, they need 
to be sure that the damaged line will not become energized again.


It's hard to build in redundancy when the entirety of lower Manhattan 
was under water though. Dealing with that must have been a hellacious job.


Mike

Re: [External] Re: 10g residential CPE

[Michael Thomas]

On 12/28/20 4:06 PM, Matthew Petach wrote:


I think you may have misunderstood Keith's comment about
it being "all a matter of time-frame."

He's right--when the sun consumes all the hydrogen in
the hydrogen-to-helium fusion process and begins to
expand into a red dwarf, that's it; there's no going
backwards, no putting the genie back into the bottle,
no "renewing" the sun.  It's purely a one-way trip.

Now, as far as humans go, we're far more likely to be
extinct due to other reasons before we come anywhere
near to that point.

But as far as the physics goes, the conversion of biomatter
into petrochemicals in the ground is more "renewable" than
the conversion of hydrogen into helium in the sun.

It's just that we're far more likely to hit the near-term
shortage crunch of petrochemicals in the ground than
we are the longer-term exhaustion of hydrogen in the
core of the sun.   ;)


2020: Hawking Radiation, take me away.

Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/29/20 8:42 AM, Aaron Wendel wrote:
Oh, we still get calls about speed issues. It's always wonderful when 
someone puts their own 10 year old Linksys WRT54G and double NATs 
behind our CPE then sends in a speed test wondering why they're only 
getting 10Mbits on their Gbit line.  We get those ALL the time. :)


Does your CPE not have wireless? If it's double NAT'ing it's at least a 
router. If it doesn't have wireless, wouldn't it be cheaper to add it so 
you don't get the support calls?


Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/29/20 9:00 AM, Mike Hammett wrote:
People love throwing their own router behind whatever Internet 
connection they have. It almost never fails to cause a problem.



Well *some* of us know what we're doing. And in my case, it's both 
because it doesn't deal with buffer bloat, but more importantly doesn't 
have wifi. I did get them to put it in bridge mode so it doesn't double nat.



Mike

Re: 10g residential CPE

[Michael Thomas]

On 12/29/20 10:36 AM, Aaron Wendel wrote:
It does have wireless.  That doesn't prevent people from trying to use 
their old equipment in addition. ("My dad's uncle's cousin's former 
roommate works in IT and told me I just needed to plug my old router 
into your new router.")




Yes, but does your CPE buffer bloat avoidance? Latency is still an issue 
when you have a big long packet queue...


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/2/21 12:40 PM, Sabri Berisha wrote:

- On Jan 1, 2021, at 2:12 PM, Matt Hoppes mattli...@rivervalleyinternet.net 
wrote:

Hi,


How would that even work?  Force a pop up into web traffic?  What if the end
users is using an app on a phone?

Most, if not all, mobile devices connected to cellular already have that 
option. On
my iphone it's under settings->notifications->government alerts. There are three
separate options: Amber alerts, Emergency alerts, and Public Safety alerts.
I wonder if that is wired in with the earthquake alerts i think you can 
get these days.


Personally, I have all three turned off after receiving nonsens alerts. Amber 
alerts
for children abducted in Los Angeles, only 600km (~450 miles) from the Bay Area,
where I live, for example. Or a "public safety" alert telling me that there are
too many people in the local Trader Joe's, 2 miles from my home.

Aliens always invade New York, so I'm safe up here :)

I beg your pardon, they have a bizarre fascination with Golden Gate 
Bridge. It's the oversized monkeys that like New York.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 12:26 AM, Valdis Klētnieks wrote:

On Sun, 03 Jan 2021 18:59:37 +1300, Mark Foster said:


In my mind it's simple.� The streaming companies need to have a channel
within their streaming system to get a message to a 'currently active
customer' (emergency popup notification that appears when their app is
open or their website is active with an authenticated user).� The

Oh geez. Just on my PS4, there's streaming apps for Disney+, Netflix, Hulu,
Prime, Playstation Store, Peacock, Tubi, ESPN+, AppleTV, YouTube (less than
half of which I actually subscribe to, but I haven't found a big enough crowbar
to remove the others, they keep returning) - and that's probably not a complete
list.

It also begs the question of what constitutes a "streaming service". Is 
my marketing department's slick new ad campaign video a "streaming 
service"? I could easily get engrossed in its valuable messaging and 
miss that tornado alert.


Inline messaging is a complete dead end on the internet. Inline was 
because there was no other reasonable way to message on broadcast tv. 
That is definitely not the case now.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/2/21 10:15 PM, b...@theworld.com wrote:

Let's just go back to air-raid sirens.

I'm old enough to remember when they were tested every day at noon,
which also told you it was noon (lunch!)

We'd say heaven help us if The Enemy attacked at noon.

They still do in San Francisco garbled message and all as it echos 
through the streets.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/2/21 10:31 PM, Jay R. Ashworth wrote:



including foreign locations, generations of emergency alert
packets *MUST* be responsibility of *LOCAL* ISPs.

A problem is that home routers may filter the broadcast
packets from ISPs, but the routers may be upgraded or
some device to snoop the alert packets may be placed between
ISPs and the routers.

Yup; it's messy, and in many many different ways.  Won't be a snapshot
rollout.  Not a bad idea, though, if implemented correctly; time to dig
out my notes, I guess.



Is there a reason not to use an outbound tcp/quic connection? It was 
unthinkable years ago to use TCP with DNS, but now we have DoH and the 
world hasn't spiraled out of control. Heck if you made it a websocket 
you'd have a built in channel for multi-media html, etc. That is, just 
push a URL down and fire up a webview that the OS makes certain is in focus.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 10:01 AM, Rich Kulawiec wrote:

On Sun, Jan 03, 2021 at 03:26:07AM -0500, Valdis Kl??tnieks wrote:

Meanwhile, this causes yet another problem - if Hulu has to be able to
know what alerts should be piped down to my device, this now means that
every single police and public safety agency has to be able to send the
alerts to Hulu (and every other streaming company) - and do this securely.

And then there's another problem (that I'm going to bet you've already
thought of, given what you've written here): Hulu and every other
streaming company need to be able to authenticate the alerts from
all those different agencies.  Those agencies also need to secure
their sending infrastructure...and good luck with that.

And then there's another problem, which is that once all those different
agencies have this facility, they're going to (ab)use it as they see fit.
I've noticed that over the last decade or so that weather alerts I've
received are covering increasingly-less-severe events, e.g., we've
slowly gone from "there's a tornado on the ground" to "there's going
to be a thunderstorm".  And at this particular point in history, I can
think of one person who would be using this every five minutes simply
because it's there.

---rsk


One of the things that makes this challenging is that not all alerts are 
created equal. I just checked and the California earthquake alert system 
is now live. For that you have maybe 10 seconds so it needs to be hella 
fast at relaying the information. Other alerts are less strict, but 
still have a real time components like the tornado alerts. And then 
there are things that effectively have no real time component like Amber 
alerts.


I'm curious how they built this:

https://earthquake.ca.gov/

Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 12:11 PM, Jay R. Ashworth wrote:

- Original Message -

From: "Michael Thomas" 
To: nanog@nanog.org
On 1/2/21 10:31 PM, Jay R. Ashworth wrote:

Yup; it's messy, and in many many different ways.  Won't be a snapshot
rollout.  Not a bad idea, though, if implemented correctly; time to dig
out my notes, I guess.

Is there a reason not to use an outbound tcp/quic connection? It was
unthinkable years ago to use TCP with DNS, but now we have DoH and the
world hasn't spiraled out of control. Heck if you made it a websocket
you'd have a built in channel for multi-media html, etc. That is, just
push a URL down and fire up a webview that the OS makes certain is in focus.

Well, TCP means that the servers have to expect to have 100k's of open
connections; I remember that used to be a problem.

As for D'oH, sure; let's centralize the attack surface.

The only reason I bring up DoH is because now there are tcp connection 
when the day before there were none. I haven't noticed any difference 
since firefox turned it, so they obviously figured out the scaling.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 1:22 PM, Mark Delany wrote:


Even with a participating application, quiescing in-memory state to something 
less than,
say, 1KB is probably hard but might be doable with a participating TLS library. 
If so, a
million quiescent connections could conceivably be stashed in a coupla GB of 
memory. And
of course if you're prepared to wear a disk read to recover quiescent state, 
your
in-memory cost could be less than 100 bytes allowing many millions of quiescent
connections per server.


Even at 1000 bytes, we're talking about 40GB for the entirety of 
California. You can get off the shelf cloud VM's with that easily these 
days, and 10 of those covers the US (ok, redundancy, but still...). 
That's probably why DoH wasn't a big deal. Throwing memory at a problem 
these days is probably easier than any heroic measures.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 1:50 PM, Mark Delany wrote:

On 03Jan21, Brandon Martin allegedly wrote:

I was thinking more in the original context of this thread w.r.t.
potential distribution of emergency alerts.  That could, if
semi-centralized, easily result in 100s of million connections to juggle
across a single service just for the USA.  While it presumably wouldn't
be quite that centralized, it's a sizable problem to manage.

Indeed. But how do you know the clients are still connected? And if they 
aren't, there is
not much a server can do beyond discarding the state. Presumably the client 
would need to
run a fairly frequent keep-a-live/reconnect strategy to ensure the connection 
is still
functioning.

Which raises the question: how long a delay do you tolerate for an emergency 
alert? I
think the end result is a lot of active connections and keep-a-live traffic. 
Not really
quiescent at all. In the end, probably just as cheap to poll a CDN.

I just sent some mail to the myshakes folks at UCB asking if they have 
an achitecture/network document. In their case for earthquakes it need 
to be less than ~10 seconds so they are really pushing the limit. If 
they get back to me, I'll share it here.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 2:23 PM, Jay R. Ashworth wrote:

- Original Message -

From: "Michael Thomas" 

Well, TCP means that the servers have to expect to have 100k's of open
connections; I remember that used to be a problem.

As for D'oH, sure; let's centralize the attack surface.

The only reason I bring up DoH is because now there are tcp connection
when the day before there were none. I haven't noticed any difference
since firefox turned it, so they obviously figured out the scaling.

Firefox is using one TCP connection to pipeline all the D'oH queries down?


I assume so. DoH is just http running http2 or http3. Clearly getting 
servers to support millions of http sessions is doable these days.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 2:27 PM, Ask Bjørn Hansen wrote:

On Jan 3, 2021, at 13:57, Michael Thomas  wrote:


I just sent some mail to the myshakes folks at UCB asking if they have an 
achitecture/network document. In their case for earthquakes it need to be less 
than ~10 seconds so they are really pushing the limit. If they get back to me, 
I'll share it here.

The two platforms they support have APIs and infrastructure to make it work at 
large scale.


Do you know where to find docs on it? I'd be curious because clearly 
this is a hard problem.




Piggybacking this sort of thing on another connection is trading some 
connection overhead for a whole lot of application complexity. This being nanog 
it’s unsurprising that the discussion is focusing on the connection and 
protocol bits, but those are a tiny part of the overall complexity (for the 
client, too). 🙂

Well, the network is an interesting part in its own right because of the 
latency is so critical. I did notice that at least part of their sensor 
network is your phone itself.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 5:00 PM, Keith Medcalf wrote:

I think the challenge here is that there's a category of people
who don't have cell phones, who don't have cable TV, but
receive content over their internet connection.  I happen to
live with someone like that, so I know it's a non-zero portion
of the population.

I pay for my Internet connection and I do not want "your shit" to be spending "my 
money".  If you think this is oh so important then *YOU* can pay to install at your sole 
expense, a device which emits your silly warnings -- I do not want them.  You will also have to 
negotiate for easement rights on my Private Property and those are not going to be given away for 
cheap.


You pay your money to your ISP and your money ceases to give a shit what 
you want. Don't like it? Give it to somebody else who does.


Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/4/21 6:44 AM, Mike Hammett wrote:
What makes the most sense is the underlying OS does the work and not 
each individual app.


The underlying OS gets these alerts from some aggregator that collects 
this information from all jurisdictions.


Doing it at the app layer seems foolish.

That probably makes sense generally, but for things like earthquakes 
which have tight requirements (= < 10 seconds), you probably need 
specialized apps.



Mike

Re: NDAA passed: Internet and Online Streaming Services Emergency Alert Study

[Michael Thomas]

On 1/3/21 2:01 PM, Andy Brezinsky wrote:
At this point I would assume that nearly every device is persisting at 
least one long lived TCP connection.  Whether it's for telemetry or 
command and control, everything these days seems to have this 
capability.  As an example, I can hit a button in the Nintendo Switch 
parent app on my phone and my kid's Switch is reflecting changes a 
second later.  That's not even a platform I would have expected to 
have that capability.


If they have an existing connection then there lots of high connection 
count solutions in the IOT space that could easily handle this number 
of connections.  A single 12c 32G box running emqttd could handle 1.3M 
connections.  Just picking a random AWS EC2 size machine, m5.4xlarge, 
would run you about $0.003/year per device to keep that connection 
open and passing data.  I assume you could drive that down 
significantly from there.


These days I would expect that just about everything has a websocket. I 
expect that google docs inspired a generation of applications.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 5:42 AM, sro...@ronan-online.com wrote:

While Amazon is absolutely within their rights to suspend anyone they want for 
violation of their TOS, it does create an interesting problem. Amazon is now in 
the content moderation business, which could potentially open them up to 
liability if they fail to suspend any other customer who hosts objectionable 
content.

When I actively hosted USENET servers, I was repeatedly warned by in-house and 
external counsel, not to moderate which groups I hosted based on content, less 
I become responsible for moderating all groups, shouldn’t that same principal 
apply to platforms like AWS and Twitter?



Is it content moderation, or just giving the boot to enabling criminal 
activity? Would that more providers be given the boot for enabling voice 
spam scams, for example. Didn't one of the $n-chan's get the boot a 
while back? I don't seem to recall a lot of push back about that and it 
was pretty much the same situation, iirc.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 9:36 AM, William Herrin wrote:

First, this would appear to be an illustration of the single-vendor
problem. You don't have a credible continuity of operations plan if a
termination by a single vendor can take you and keep you offline. It's
the single point of failure that otherwise intelligent system
architects fail to consider and address. But more than that, cloud
providers like Amazon tend to make it inconvenient approaching
impossible to build cross-platform services. I kinda wonder what a
cloud services product would look like that was actively trying to
facilitate cross-platform construction?


I suppose it depends on how distributed your system design is. You 
certainly don't want to be running low latency necessary storage in one 
provider and servers in another. But you certainly need to architect for 
multi-region, and it seems to me that's the place to make the cut for 
cross provider as well. But AWS does have one incentive on the 
networking front: they want to peel off computing with corpro data 
centers which means they need to integrate with high speed vpn's and the 
like. Maybe somebody knows whether the likes of AWS and others are 
considered to be inside the corpro perimeter and how that works in a 
multi-tenancy world.




Second, Amazon strongly encourages customers to build use of its
proprietary services and APIs into the core of the customer's product.
That's quite devastating when there's a need to change vendors.
Parler's CEO described Amazon's action as requiring them to "rebuild
from scratch," so I wonder just how tightly tied to such Amazon APIs
they actually are. And if there isn't a lesson there for the rest of
us.


Yes, it's been obvious to anybody who's only paying even a little 
attention that AWS is trying to be build a walled garden. It always 
surprises me how little people take into consideration that that almost 
never ends well for the people lured into the garden. As it ever were, I 
guess. I guess the lesson is that if you're sketch consider portability. 
If you're not sketch, consider portability anyway.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 9:55 AM, Töma Gavrichenkov wrote:

Peace,

On Sun, Jan 10, 2021 at 8:38 PM William Herrin  wrote:

providers like Amazon tend to make it inconvenient approaching
impossible to build cross-platform services. I kinda wonder what a
cloud services product would look like that was actively trying to
facilitate cross-platform construction?

HashiCorp Nomad plus Terraform.  That's pretty much it.

I'd say it starts to be "inconvenient approaching impossible" only at
the point where you begin to use Cloudformation — or when you don't
have automated deployment at all.  While the provisioning tools are
provider agnostic, a move from a provider to a provider would take
days at most.

At my previous job, I built a tool which could spin up a server farm 
given a platform agnostic design spec from a list of vendors as well as 
pricing it out. It was really more of a prototype since it only 
supported Chef on the spin-up side, but it showed that you could move 
things pretty quickly if need be. I hadn't considered this as a use case 
though.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 10:17 AM, Töma Gavrichenkov wrote:

Peace,

On Sun, Jan 10, 2021 at 9:09 PM Michael Thomas  wrote:

Yes, it's been obvious to anybody who's only paying even a little
attention that AWS is trying to be build a walled garden.

In my experience, moving off Amazon services isn't that much of a
trouble, especially if compared to moving off Azure.  Cloud Active
Directory + Sentinel + PowerBI — and boom, your company is with Azure
for life.

To be fair, they all want walled gardens. And there are plenty of junior 
engineers attracted to shiny new things that people need to keep an eye on.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 10:24 AM, Töma Gavrichenkov wrote:

Peace,

On Sun, Jan 10, 2021 at 9:18 PM Michael Thomas  wrote:

At my previous job, I built a tool which could spin up a server farm
given a platform agnostic design spec from a list of vendors as well as
pricing it out. It was really more of a prototype since it only
supported Chef on the spin-up side, but it showed that you could move
things pretty quickly if need be.

That's not just the provisioning, you also need an independent cluster
management and scheduling (hence Nomad).  But yes.

The real point of the tool was to capture all of those architectural 
features so they could be compared and contrasted. Some of them are 
fungible, some of them are not. Knowing the non-fungible ones gives you 
a window into lock ins.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 10:21 AM, William Herrin wrote:

On Sun, Jan 10, 2021 at 9:55 AM Töma Gavrichenkov  wrote:

I'd say it starts to be "inconvenient approaching impossible" only at
the point where you begin to use Cloudformation — or when you don't
have automated deployment at all.  While the provisioning tools are
provider agnostic, a move from a provider to a provider would take
days at most.

Hi Töma,

Are you sure about that? Consider your database. Suppose you want to
run your primary database in AWS with a standby replica in Azure. As
long as you install your own database software in both, you can do
that. But if you want to leverage AWS' RDS products too, you're mostly
out of luck.


Is RDS based on something else? I find it hard to believe that they 
wrote a rdb from scratch. But yes, once they own your db they own you. 
I've looked before how to migrate from mysql to postgres and was shocked 
at how little there seems to be out there to even do even the easier 
stuff let alone the proprietary extensions.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 11:11 AM, Bryan Fields wrote:


Anyone hosting with Amazon/Google/the cloud here should be really concerned
with the timing they gave them, 24 hours notice to migrate.  Industry
standards would seem to be at least 30 days notice.  Note this is not the
police/courts coming to the host with notice that they are hosting illegal
content but only the opinion of the provider that they don't want to host it.

Considering that it seems that there continues to be talk/planning of 
armed insurrection, I think we can forgive them for violating 
professional courtesy.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 12:13 PM, Matthew Petach wrote:



On Sun, Jan 10, 2021 at 12:03 PM Michael Thomas <mailto:m...@mtcc.com>> wrote:



On 1/10/21 11:11 AM, Bryan Fields wrote:
>
> Anyone hosting with Amazon/Google/the cloud here should be
really concerned
> with the timing they gave them, 24 hours notice to migrate. 
Industry
> standards would seem to be at least 30 days notice. Note this is
not the
> police/courts coming to the host with notice that they are
hosting illegal
> content but only the opinion of the provider that they don't
want to host it.
>
Considering that it seems that there continues to be talk/planning of
armed insurrection, I think we can forgive them for violating
professional courtesy.

Mike


I thought the boot was announced after physical threats were made
against Google and Apple facilities and employees for removing the
app from the app stores?

There's professional courtesy; but the moment you start threatening
to bomb datacenters and kill employees, it's pretty clear professional
courtesy has been forcibly thrown through the reinforced double-glazed
energy-efficient windows and has plummeted straight through the roof
of the classic Cadillac in the parking lot ten stories below.  :(

Yeah, it may have been self-interest or all of the above. I really don't 
think that much concern needs to be given any more than the concern 
given to spammers and other fraudsters. "you have 30 days to move your 
criminal enterprise, have a good day". Bletch.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 3:15 PM, Izaac wrote:

On Sun, Jan 10, 2021 at 12:01:46PM -0800, Michael Thomas wrote:

Considering that it seems that there continues to be talk/planning of armed
insurrection, I think we can forgive them for violating professional
courtesy.

Got links?


Ask Google, Apple and Amazon. I'm sure they have the receipts.

Mike

Re: Parler

[Michael Thomas]

On 1/10/21 4:00 PM, Eric S. Raymond wrote:

sro...@ronan-online.com :

While Amazon is absolutely within their rights to suspend anyone they want for 
violation of their TOS, it does create an interesting problem. Amazon is now in 
the content moderation business, which could potentially open them up to 
liability if they fail to suspend any other customer who hosts objectionable 
content.

When I actively hosted USENET servers, I was repeatedly warned by in-house and 
external counsel, not to moderate which groups I hosted based on content, less 
I become responsible for moderating all groups, shouldn’t that same principal 
apply to platforms like AWS and Twitter?

Yes, it would.  This was an astonnishingly stupid move on AWS's part;
I'm prett sure their counsel was not conmsulted.



I dunno, maybe think of it as a good behavior discount coupon. They're 
free to build out their own datacenters at their cost if they don't get 
that coupon. And them not talking to council is what would be 
astonishing. Mostly likely it's been on all of their radars for quite 
some time. It's not like it was a secret that it was an open sewer.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 3:40 PM, Izaac wrote:

On Sun, Jan 10, 2021 at 03:36:18PM -0800, Michael Thomas wrote:

On Sun, Jan 10, 2021 at 12:01:46PM -0800, Michael Thomas wrote:

Considering that it seems that there continues to be talk/planning of armed
insurrection, I think we can forgive them for violating professional
courtesy.

Got links?

Ask Google, Apple and Amazon. I'm sure they have the receipts.

No, you made this claim.  I am asking you.  Got links?  Screenshots?

I didn't realize this was a high school debate.  Your beef is with them, 
not me.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 4:48 PM, Dan Hollis wrote:

On Sun, 10 Jan 2021, Michael Thomas wrote:

On 1/10/21 3:15 PM, Izaac wrote:

On Sun, Jan 10, 2021 at 12:01:46PM -0800, Michael Thomas wrote:
Considering that it seems that there continues to be talk/planning 
of armed

insurrection, I think we can forgive them for violating professional
courtesy.

Got links?

Ask Google, Apple and Amazon. I'm sure they have the receipts.


You made the claim.

I also said "seems" which means that I won't be bothered dredging up 
documentation for something easily looked up by people trying to excuse 
the inexcusable. I've seen bits and pieces from the open sewer. That is 
quite enough.


Mike

Re: Parler

[Michael Thomas]

On 1/10/21 10:33 PM, Randy Bush wrote:

In article <474fe6a6-9aa8-47a7-82c6-860a21b0e...@ronan-online.com> you write:

When I actively hosted USENET servers, I was repeatedly warned by in-house and 
external counsel, not to moderate which groups I hosted
based on content, less I become responsible for moderating all groups, 
shouldn’t that same principal apply to platforms like AWS and
Twitter?

If this was in the US and it was after the CDA was passed in 1996,
your lawyers were just wrong.

it is really annoying that you leave not the slightest clue to who the
hell you are replying


+1

Mike

Re: Parler

[Michael Thomas]

On 1/10/21 9:01 PM, William Herrin wrote:

Look closer. The AWS RDS version of mysql is unable to replicate with
your version of mysql. The configuration which would permit it is not
exposed to you.

Unless something has changed in the last couple years?


Anything that abstracts database services screams LOCK IN. Does anybody 
know what the supposed value of this lock in is?


Mike, wondering whether there exists database repair people like when my 
toilet explodes

Re: DoNotPay Spam?

[Michael Thomas]

oh is that where it's coming from. yes. my filter now zaps it.

Mike

On 1/13/21 2:06 PM, Robert Webb wrote:
Anyone else getting spam from DoNotPay everytime they send an email to 
the list?


I have not sent anything in a while until my ATT email and now I am 
getting this on every new email I send to the list.


Alternate text

**


*You’re almost there! Sign up once to unlock lifetime protection (and 
even compensation) on all spam emails. *

Re: Hosting recommendations ... ?

[Michael Thomas]

On 1/19/21 9:33 AM, Brandon Martin wrote:

On 1/19/21 11:44 AM, William Herrin wrote:

Cloud = you get virtual servers with virtual storage, generally
adjustable to meet your needs. You manage the operating systems and
storage within the virtual environment. You DO NOT manage the host
operating systems or hypervisors.


It's worth pointing out that nested virtualization is a thing these 
days, and some providers might even support it!  That means you could 
buy one large instance and sub-divide it yourself into multiple VMs if 
you want to.


In practice, unless you need that flexibility to dynamically spin the 
VMs up and down with various specs AND don't want to or cannot use a 
provider's API for that, I'm not sure why you'd want to if you didn't 
have to for some crazy reason.



Except for the problem that you pay a premium for larger vm's. you can't 
compete with t instances on aws, for starters.


Mike

Re: DoD IP Space

[Michael Thomas]

On 2/11/21 5:41 PM, Izaac wrote:



IPv6 restores that ability and RFC-1918 is a bandaid for an obsolete protocol.

So, in your mind, IPv4 was "obsolete" in 1996 -- almost three years
before IPv6 was even specified?  Fascinating.  I could be in no way
mistaken for an IPv4/NAT apologist, but that one's new on me.


ipv6 was on my radar in the early 90's. it was definitely at least 1993, 
maybe earlier.


Mike

Re: Texas internet connectivity declining due to blackouts

[Michael Thomas]

On 2/16/21 3:05 AM, Jared Mauch wrote:
Almost exactly 4 years ago we were out up here in Michigan for over 
120 hours after a wind storm took out power to 1 million homes. Large 
scale restoration takes time. When the load and supply are imbalanced 
it can make things worse as well.


I'm hoping things return to normal soon but also am reminded it can 
take some time.


We now have a large generator with automatic switchover after that 
event. Filling gas cans every 12 hours to feed the generator is no fun.


We use propane. It's less dense energy-wise than gasoline, but it's 
really easy to switch over.


Mike

Re: Texas internet connectivity declining due to blackouts

[Michael Thomas]

On 2/16/21 8:50 AM, John Von Essen wrote:
I just assumed most people in Texas have heat pumps- AC in the summer 
and minimal heating in the winter when needed. When the entire state 
gets a deep freeze, everybody is running those heat pumps non-stop, 
and the generation capacity simply wasn’t there. i.e. coal or natural 
gas plants have some turbines offline, etc.,. in the winter because 
historically power use is much much less. The odd thing is its been 
days now, those plants should be able to ramp back up to capacity - 
but clearly they haven’t. Blaming this on wind turbines is BS. In 
fact, if it weren’t for so many people in Texas with grid-tie solar 
systems, the situation would be even worse.


You'd think that mid-summer Texas chews a lot more peak capacity than 
the middle of winter. Plus I would think a lot of Texas uses natural gas 
for heat rather than electricity further mitigating its effect on the grid.


Mike

dumb question: are any of the RIR's out of IPv4 addresses?

[Michael Thomas]

Basically are there places that you can't get allocations? If so, what 
is happening?


Mike

Re: Texas internet connectivity declining due to blackouts

[Michael Thomas]

On 2/16/21 3:19 PM, Sabri Berisha wrote:

- On Feb 16, 2021, at 6:28 AM, Michael Thomas m...@mtcc.com wrote:


We use propane. It's less dense energy-wise than gasoline, but it's
really easy to switch over.

Why not use both? Plenty of generators that are dual fuel out there.
Last year I converted my Duramax to dual fuel by replacing the
carburator. Easy-peasy.


gasoline has a shelf life, though with PG&E that isn't a problem :/

but the larger issue is that i really would prefer not have a bunch of 
gasoline around. it's messier too in comparison to just switching a 
propane tank. we have like three or four 5 gallon tanks which we use in 
the mean time for bbq's, etc. we manage to run the things we need for 
about 24 hours on one tank.


Mike

Re: dumb question: are any of the RIR's out of IPv4 addresses?

[Michael Thomas]

On 2/16/21 4:18 PM, Fred Baker wrote:
You may find this article interesting: 
https://blog.apnic.net/2019/12/13/keep-calm-and-carry-on-the-status-of-ipv4-address-allocation/ 
<https://blog.apnic.net/2019/12/13/keep-calm-and-carry-on-the-status-of-ipv4-address-allocation/>


So aside from Afrinic, this is all being done on the gray market? 
Wouldn't you expect that price to follow something like an exponential 
curve as available addresses become more and more scarce and unavailable 
for essentially any price?


Mike



Sent from my iPad


On Feb 16, 2021, at 3:07 PM, Michael Thomas  wrote:


Basically are there places that you can't get allocations? If so, 
what is happening?


Mike

Re: Texas internet connectivity declining due to blackouts

[Michael Thomas]

On 2/17/21 7:15 AM, Sean Donelan wrote:


The price of electricity is a major component of the decision where 
data centers operators choose to build large data centers.



Total electric price to end consumer (residential).  Although 
industrial electric prices are usually lower, its easier to compare 
residential prices across countries.


Europe (Residential):
Lowest Bulgaria: EU 9.97 cents/kWh (USD 12.0 cents/kWh)
Highest Germany: EU 30.88 cents/kWh (USD 37.33 cents/kWh)

Average: EU 20.5 cents/kWh (USD 25.2 cents/kWh)

USA (Residential):
Lowest Idaho: USD 9.67 cents/kWh (EU 8.3 cents/kWh)
Highest Hawaii: USD 28.84 cents/kWh (EU 24.07 cents/kWh)

Average: USD 13.25 cents/kWh (EU 10.79 cents/kWh)


Texas is slightly below the US average at
Texas: USD 12.2 cents/kWh (EU 9.96 cents/kWh)



here in California it's like $.20 - $.30 with pg&e. i recently looked up 
Oregon and it was like $.03 which is why you probably see data centers 
being built by The Dalles and Prineville.


Mike

Re: Texas internet connectivity declining due to blackouts

[Michael Thomas]

On 2/17/21 9:40 AM, Aaron C. de Bruyn via NANOG wrote:
It might not be an easy fix in the moment, but in the long run, buy a 
generator and install a propane tank.
When power prices spike to insane levels like this, just flip your 
transfer switch over and run off propane.

When utility power becomes cheaper, switch back to the grid.

Maybe some sort of Raspberry Pi to monitor the current prices and do 
the transfer automatically.  (language warning: 
https://www.youtube.com/watch?v=gz7IPTf1uts 
)


Protip: If you're blacked out, it doesn't matter what the price of 
power is.


We just run extension cords and don't have a transfer switch. It's 
pretty surprising what you can run on about a kw. A gallon propane tank 
lasts close to 24 for us.


Mike

Re: Texas internet connectivity declining due to blackouts

[Michael Thomas]

On 2/17/21 1:23 PM, b...@uu3.net wrote:

Hold on.. Math doesnt add-up here.
Are you telling me that a gallon propane tank (3.8l) can last
24 hours for about 1000W power generation. Are you sure?
I could belive for 6 hours... maybe 8.. not 24 hours.
So either you are using up 200-300W.. or you have superior power
generator. Can you share what are you using?


Sorry I noticed my error right after I hit send. I meant a 5 gallon 
tank, not 1. Inverter generators are definitely worth the extra cost though.


Mike




-- Original message --

From: Michael Thomas 
To: nanog@nanog.org
Subject: Re: Texas internet connectivity declining due to blackouts
Date: Wed, 17 Feb 2021 09:56:06 -0800

We just run extension cords and don't have a transfer switch. It's pretty
surprising what you can run on about a kw. A gallon propane tank lasts close to
24 for us.

Mike

Re: Texas internet connectivity declining due to blackouts

[Michael Thomas]

On 2/17/21 2:37 PM, Carsten Bormann wrote:


I actually tend to believe that buried HVDC is the future of long-distance 
power transmission.
We might be able to pull off that this transitions from a niche technology to 
the mainstream, like we did with photovoltaics (at the cost of 200 G€).
Let’s see...

I wonder what a world with 5V DC distributed within the house would look 
like. All of those power adapters are both ugly and a PITA. Of course 
that wouldn't have to come in from the grid, but still. I found a 
powerstrip which has a couple of USB slots in it and it's very nice. It 
also allows the AC plugs to be rotated which is nice for the remaining 
adapters.


Mike

Re: OT: Re: Facebook and other walled gardens

[Michael Thomas]

On 3/22/21 9:02 AM, Grant Taylor via NANOG wrote:

On 3/22/21 8:00 AM, Mike Hammett wrote:

most discussion in the WISP space has moved to Facebook


So ... a walled garden.

I have a severe problem with professional communities /requiring/ me 
to have a Facebook, et al., account to participate in community 
discussions.


What am I supposed to do if I don't have / can't get / want to avoid 
$WalledGardenInQuestion?  Am I forced to choose to break down and get 
an account with $WalledGardenInQuestion XOR not participate in said 
professional community?


What happens when $WalledGardenInQuestion changes policies or 
otherwise disagrees with something or is decides to ban the country 
where I'm at?


I think that such walled gardens are a Bad Idea™.



That's especially true since their moderation AI's are terrible and 
arbitrary and absolutely not up to the task. Getting put into FB jail 
for something outside of work affecting you at work is not OK. And since 
they use browser fingerprinting, etc, having two separate account while 
in FB jail looks like ban evasion to them and will get you site banned.


Mike

Re: OT: Re: Facebook and other walled gardens

[Michael Thomas]

On 3/22/21 11:41 AM, William Herrin wrote:

On Mon, Mar 22, 2021 at 10:23 AM Andy Ringsmuth  wrote:

No. Use a communication method that is available globally, not proprietary and 
doesn’t require me to sell my soul to the devil simply to participate.

Hi Andy,

I refused to get a Facebook account until I was paid to. Now that I
have one, I wonder why I bothered. I isolate it in its own browser
profile so it can't snoop the rest of my web activity and I gave it an
alias email address that only they have. I mostly  control what
information I give them. I like having an effortless way to keep up
with my extended friends and family. In spite of that, I was surprised
how good a job Facebook did targeting ads to my interests -- the
knight hoodies were just too cool.


Air-gapping the browsers is certainly best, but Firefox has been really 
clamping down on super-cookies and the like. It will be interesting to 
see what the overall effect is.


My husband and friends have been having good laughs lately about some of 
the weird targeted ads they get. They are definitely not perfect. I use 
Facebook Purity and don't see their ads at all.


Mike

Re: OT: Re: Younger generations preferring social media(esque) interactions.

[Michael Thomas]

On 3/22/21 11:22 PM, Cynthia Revström via NANOG wrote:

Hi,

As someone from a "younger generation" (2001) who does use mailing 
lists, semi-actively participates in RIPE mailing lists but also 
created a network community on Discord, I want to chime in here.


> Are they willing to use a (traditional) forum (of sorts) that is
dedicated to the venue? Or Are they wanting things to come to them
wherever they happen to be today?  E.g. Facebook group, Discord, 
Slack, etc?


I haven't ever used facebook beyond receiving some invitation for an 
event, and I feel like that's the most common case for people around 
my age group. (not using Facebook that is)


I'm under the impression that for the younger generations that Facebook 
is deeply uncool. It's where grandma posts pictures of her knitting.


Mike

Re: OT: Re: Younger generations preferring social media(esque) interactions.

[Michael Thomas]

On 3/23/21 1:44 AM, Mikael Abrahamsson via NANOG wrote:

On Mon, 22 Mar 2021, Grant Taylor via NANOG wrote:

If it's the latter, does that mean that you have to constantly keep 
changing /where/ messages are sent to in order to keep up with the 
latest and greatest or at least most popular (in your audience) 
flavor of the day / week / month / year social media site?


All good questions. I've been using IRC+email for 25+ years now and 
from what I can see, IRC has been replaced by slack/discord etc, and 
email has been replaced by Reddit or Github Issues discussions etc. I 
was on a project where the mailing list was shut down and all further 
discussions were pushed to github instead.


I personally think the "web forum" format is inferior but that might 
be a way to reach out as well...


The big problem with mailing lists is that they screw up security by 
changing the subject/body and breaking DKIM signatures. This makes 
companies leery of setting the signing policy to reject which makes it 
much easier for scammers to phish. The Nanog list is something of an 
outlier in that they don't do modifications and the DKIM signature survives.


I wrote a piece about this a while back that companies should just set 
p=reject and ignore the mailing list problem.


https://rip-van-webble.blogspot.com/2020/12/are-mailing-lists-toast.html

Mike

Re: OT: Re: Younger generations preferring social media(esque) interactions.

[Michael Thomas]

On 3/23/21 2:55 PM, Grant Taylor via NANOG wrote:

On 3/23/21 1:40 PM, Michael Thomas wrote:
The big problem with mailing lists is that they screw up security by 
changing the subject/body and breaking DKIM signatures.


What you are describing is a capability, configuration, execution 
issue with the mailing list manager software.


Said another way, what you are describing is *NOT* a problem with the 
concept of mailing lists.


MLMs can easily receive messages -- after their MTA imposes all 
germane filtering -- and generate /new/ but *completely* *independent* 
messages substantially based on the incoming message's content.  These 
/new/ messages come /from/ /the/ /mailing/ /list/!  Thus the mailing 
list operators can leverage all the aforementioned security / safety 
measure for the mailing list.
But they still have the originating domain's From: address. Manifestly 
using MLM signatures as means of doing a reputation check is a 
previously unsolved problem hence the silliness of the ARC experiment 
which relies on the same assumption you are making here. Since Google 
participated in ARC, that is a pretty tacit admission they don't know 
how to do mailing list reputation either.


SPF / DKIM / DMARC are mean to enable detection (and optionally 
blocking) of messages that do not come from their original source. 
Mailing lists are inherently contrary to this.  But the mailing list 
can be a /new/ source.
The sticking point is the From: address. If I set up a DMARC p=reject 
policy, I should not be surprised that the receiver does what I asked 
and trashes mailing list traffic. The point in my blog post is that 
after over 15 years a solution is not going to be found, and trust me I 
have tried back in the day. That we should just give up caring about 
mailing list traversal and put the burden on MLM's to figure it out by 
either not changing the message body/subject, or using that horrible 
hack of rewriting the From address.


This makes companies leery of setting the signing policy to reject 
which makes it much easier for scammers to phish.


Hence, having the mailing list send out /new/ messages with /new/ 
protection measures mean less breakage for people that send messages 
to the mailing list.


Mailing lists have been sending out resigned messages for over a decade. 
We still have really low adoption of p=reject signing policy and at 
least part of the problem is because of fear of mailing lists affecting 
users.




Treating the mailing list as it's own independent entity actually 
enables overall better security.


Aside:  It is trivial to remove things that cause heartburn (DKIM) 
/after/ NANOG's SMTP server applies filtering /before/ it goes into 
Mailman.



An unsigned message is treated the same as a broken signature. That 
doesn't help from the From: signing policy standpoint.


Mike

Re: OT: Re: Younger generations preferring social media(esque) interactions.

[Michael Thomas]

On 3/23/21 4:34 PM, Grant Taylor via NANOG wrote:

On 3/23/21 4:16 PM, Michael Thomas wrote:

But they still have the originating domain's From: address.


My opinion is that messages from the mailing list should not have the 
originating domain in the From: address.  The message from the mailing 
list should be from the mailing list's domain.


This has the unfortunate downside of teaching people not to pay 
attention to the From: domain. For mailing lists maybe that's an OK 
tradeoff, but it definitely not a good thing overall. I noticed that the 
IETF list does From re-writing for DMARC domains that are p=reject.





Don't try to graft "can I trust what the mailing list purports or not" 
question onto the problem.  Simplify it to "does this message (from 
the mailing list) pass current best practice security tests or not".  
Notice how the second question is the same question that is already 
being posed about all email (presuming receiving server is doing so).



That is the essence of the problem and always has been. If somebody 
resigns an altered message, does that change my decision of what to do 
in the face of DMARC p=reject? That means I need to know something about 
that mailing list if the answer is yes. Best practices have nothing to 
do with it. It is all about reputation. A message mangler can be Lawful 
Evil, after all.




Since Google participated in ARC, that is a pretty tacit admission 
they don't know how to do mailing list reputation either.


IMHO ARC has at least a priming / boot strapping problem.  How does a 
receiver know if they can trust the purported information they receive 
from the sending system or not.  Simply put, it doesn't.  Hence why I 
think that ARC, as I understand it, is going to fail to thrive.


I went back to the DMARC mailing list wondering what magic that ARC 
provided that we didn't think about 15 years ago only to be disappointed 
that the answer was "none". I really don't understand how this got past 
IESG muster, but it was an experiment.





I personally believe that the mailing list manager, or better it's 
underlying SMTP server infrastructure, should uphold strict 
requirements on the incoming messages.  Only clean messages should be 
emitted from the mailing list manager.  Further, those messages should 
themselves adhere to the same high security standards.


Yes, I think that's a given and feeds into their reputation.




Think about it this way:  Is there really anything (of significant 
value) different between a mailing list manager and a person (or other 
form of automation) receiving a message from a mailbox, copying and 
pasting it (work with me here) into a new message and sending it 
$NumberOfSubscribers times per message to the mailing list?  --  I 
don't think there is.


From the standpoint of the receiving domain, it has no clue who mangled 
the original message. The only thing they know is that there isn't a 
valid signature from the originating domain and what the originating 
domain's advice is for that situation.





What would you want SPF / DKIM / DMARC to do if I took a message from 
you (directly vs passing through the mailing list manager) and changed 
the recipient(s) and re-sent it out to one or more other people?  -- 
I'd wager a reasonable lunch that most people would want SPF / DKIM / 
DMARC to detect and possibly thwart such forwarding.  --  So why is a 
mailing list held to different (lower) standards?


This is the so-called replay attack. It's nonsense. Email has always 
been essentially multicast.



An unsigned message is treated the same as a broken signature. That 
doesn't help from the From: signing policy standpoint.


The original From: signature should have been validated, weighted, and 
judged /before/ it made it to the mailing list manager. Further, the 
mailing list manager should have removed any reference to the original 
signature.  


Signatures shouldn't be removed: a broken signature is identical to a 
missing signature security-wise, but broken signatures can be used for 
forensics. I, for example, could reconstruct a very large percentage of 
mailing list messages to unbreak signatures. It was to the point that it 
was quite tempting to use that approach to deal with MLM traversal.


Mike

Re: OT: Re: Younger generations preferring social media(esque) interactions.

[Michael Thomas]

On 3/24/21 5:38 PM, Bryan Fields wrote:

On 3/23/21 8:04 PM, Michael Thomas wrote:

This has the unfortunate downside of teaching people not to pay
attention to the From: domain. For mailing lists maybe that's an OK
tradeoff, but it definitely not a good thing overall. I noticed that the
IETF list does From re-writing for DMARC domains that are p=reject.

This is another reason why DMARC is a shitty solution.

NANOG will rewrite the From: as well in this case.

What's your solution to phishing then? FWIW, nanog doesn't alter 
messages. All lists have the option to follow suit.


Mike

Re: OT: Re: Younger generations preferring social media(esque) interactions.

[Michael Thomas]

On 3/24/21 5:57 PM, Bryan Fields wrote:

On 3/24/21 8:44 PM, Michael Thomas wrote:

FWIW, nanog doesn't alter
messages. All lists have the option to follow suit.

It does.  There's a setting in mailman that's enabled for the nanog list.

dmarc_moderation_action (privacy): Action to take when anyone posts to the
list from a domain with a DMARC Reject/Quarantine Policy.

It's set to Munge From.

So if your domain had a reject policy on DMARC, nanog will munge the from:
header to be 'Your Name via NANOG'.

Yes, the IETF list does that, for example. But NANOG's just not 
modifying anything is better, IMO.


But the larger point is that people should set p=reject and put the 
burden on mailing lists to adapt.


Mike

10 years from now... (was: internet futures)

[Michael Thomas]

On 3/26/21 12:26 PM, Mark Tinka wrote:
If the last decade is anything to go by, I'm keen to see what the next 
one brings.


Mark.



So the obvious question is what will happen to the internet 10 years 
from now. The last 10 years were all about phones and apps, but that's 
pretty well played out by now. Gratuitously networked devices like my 
dishwasher will probably be common, but that's hardly exciting. LEO 
internet providers will be coming online which might make a difference 
in the corners of the world where it's hard to get access, but will it 
allow internet access to parachute in behind the Great Firewall?


One thing that we are seeing a revolution in is with working from home. 
That has some implications for networking since symmetric bandwidth, or 
at least quite a bit more upstream would be helpful as many people found 
out. Is latency going to drive networking, given gaming? Gamers are not 
just zitty 15 year olds, they are middle aged or older nowadays.


Mike

Re: 10 years from now... (was: internet futures)

[Michael Thomas]

On 3/26/21 2:00 PM, Mark Andrews wrote:

There are more smart phones in use in the world today the world than can be 
addressed by IPv4. Complaining about lack of IPv6 deployment has been 
legitimate for a long time. Telcos shouldn’t have to deploy NATs. Homes 
shouldn’t have to deploy NATs. Businesses shouldn’t have to deploy NATs.

NATs produce a second class Internet.  We have had to lived with a second class 
Internet for so long that most don’t know what they are missing.


I thought a fair chunk of mobile phones were using ipv6?

Mike

Re: 10 years from now... (was: internet futures)

[Michael Thomas]

On 3/26/21 3:31 PM, Mark Tinka wrote:



On 3/26/21 23:30, b...@uu3.net wrote:


Oh, sorry to disappoint you, but they are not missing anything..
Internet become a consumer product where data is provided by
large corporations similary to TV now. Your avarage Joe consumer
does NOT care about NAT and that he cant run services or he does NOT
have full e2e communication.


Yep - infrastructure is now implied, to the extent that customers even 
forget who are they paying for connectivity.




Yes, you are right, NAT was a second class internet for a while but
now it seems that we cannot live without it anymore :)
I dont really see other way how I can connect LAN to internet now.
Using public IPs? Thats so terrible idea. How can I be el-cheappo
dual-homed then?


As long as infrastructure continues to dilly-dally, software will fill 
in the gaps, even if it may cause more breakage in the eyes of the 
networking purists.



I think the question these days is NAT or not. It's double NAT or not.

Mike

Re: 10 years from now... (was: internet futures)

[Michael Thomas]

On 3/27/21 2:50 AM, Valdis Klētnieks wrote:

On Fri, 26 Mar 2021 12:42:20 -0700, Michael Thomas said:


dishwasher will probably be common, but that's hardly exciting. LEO
internet providers will be coming online which might make a difference
in the corners of the world where it's hard to get access, but will it
allow internet access to parachute in behind the Great Firewall?

At which point, we get to see two very different types of LEO engage
in mortal combat



PREEEYYY FIREWORKS!!!

I'm sure somebody's thought about this, but are these LEO networks 
intended to have the downlink at home?  How do the Chinas of the world 
intend to deal with the Great Firewall implications?


Mike

Re: 10 years from now... (was: internet futures)

[Michael Thomas]

On 3/29/21 11:36 AM, Matt Erculiani wrote:



We might be talking a lot more about PRKI as it becomes compulsory, 
maybe 400G transit links will start being standard across the 
industry. If we're lucky (or unlucky, depending on how you look at it) 
maybe a whole new routing protocol will be introduced and rapidly gain 
popularity.


One interesting observation is that QUIC has the potential to open the 
floodgates for new purpose built transport protocols for things other 
than http that have their own requirements. It also shows that it can 
navigate the problem of pleading kernel code and firewalls that block 
unknown (it it) IP protocol numbers. It's my guess that those were what 
really sunk SCTP. Another thing that is coming up is that with 
increasingly high bandwidth, the TCP checksum is showing its age and 
we'd probably like to leverage crypto-grade hashes instead of being at 
the mercy of a 40 year old algorithm.


Mike

Re: Texas ERCOT power shortages (again) April 13

[Michael Thomas]

On 4/14/21 7:00 AM, Brian Johnson wrote:
There is no profit motive for a non-profit company. It’s completely 
relevant to your response.



This is patently absurd. It's an industry group/organization. It's 
raison d'etre is to serve its industry which definitely has a profit 
motive. That and even non-profits have a profit motive to stay afloat. 
See the NRA for one that has gone terribly wrong.


Mike

Re: Malicious SS7 activity and why SMS should never by used for 2FA

[Michael Thomas]

I wonder how much of this is moot because the amount of actual SS7 is 
low and getting lower every day. Aren't most "SMS" messages these days 
just SIP MESSAGE transactions, or maybe they use XMPP? As I understand a 
lot of the cell carriers are using SIPoLTE directly to your phone.


Mike

On 4/18/21 8:24 AM, Mel Beckman wrote:
Although NIST “softened” its stance on SMS for 2FA, it’s still a bad 
choice for 2FA. There are many ways to attack SMS, not the least of 
which is social engineering of the security-unconscious cellular 
carriers. The bottom line is, why use an insecure form of 
communication for 2FA at all? Since very good hardware-token-quality 
OTP apps are freely available, why be so lazy as to implement 2FA 
using radically insecure SMS?


Your argument that 2FA is only meant to “enhance” the security of a 
memorized password is just wrong. 2FA is meant as a /bulwark /against 
passwords that very often are disclosed by data breaches, through no 
fault of the password owner. 2FA enhances nothing. It guards against 
the abject security failures of others.


Consider this sage advice from 2020, long after NIST caved to industry 
pressure on its recommendations.


https://blog.sucuri.net/2020/01/why-2fa-sms-is-a-bad-idea.html 



  -mel

On Apr 18, 2021, at 8:02 AM, William Herrin > wrote:


On Sun, Apr 18, 2021 at 7:32 AM Mel Beckman > wrote:
SMS for 2FA is not fine. I recommend you study the issue in more 
depth. It’s not just me who disagrees with you:


https://www.schneier.com/blog/archives/2016/08/nist_is_no_long.html 



Mel,

That Schneier article is from 2016. The 3/2020 update to the NIST
recommendation (four years later and the currently active one) still
allows the use of SMS specifically and the PSTN in general as an out
of band authenticator in part of a two-factor authentication scheme.
The guidance includes a note explaining the social engineering threat
to SMS authenticators: "An out of band secret sent via SMS is received
by an attacker who has convinced the mobile operator to redirect the
victim’s mobile phone to the attacker."

https://pages.nist.gov/800-63-3/sp800-63b.html#63bSec8-Table1 



The bottom line is that an out-of-band authenticator like SMS is meant
to -enhance- the security of a memorized secret authenticator, not
replace it. If properly used, it does exactly that. If misused, it of
course weakens your security.

Regards,
Bill Herrin



--
William Herrin
b...@herrin.us
https://bill.herrin.us/

Re: DoD IP Space

[Michael Thomas]

On 4/24/21 3:45 PM, William Herrin wrote:

On Sat, Apr 24, 2021 at 8:26 AM Mel Beckman  wrote:

This doesn’t sound good, no matter how you slice it. The lack of
transparency with a civilian resource is troubling at a minimum.

You do understand that the addresses in question are not and have
never been "civilian." They came into DoD's possession when this was
all still a military project funded by what's now DARPA.

Personally, I think we may have an all time record for the largest
honeypot ever constructed. I'd love to be a fly on that wall.

Is this to say that the prefixes are now being announced? Sorry for this 
dumb question, but how would this honeypot work?


Mike

Something that should put a smile on everybody's face today

[Michael Thomas]

And we can help! Cloudflare is setting out to destroy a patent troll:

https://www.techdirt.com/articles/20210426/09454946684/patent-troll-sable-networks-apparently-needs-to-learn-lesson-cloudflare-wants-to-destroy-another-troll

Mike

Re: Something that should put a smile on everybody's face today

[Michael Thomas]

Cloudflare is a service provider. Getting sued by patent trolls is an 
operational issue. And you're a fine one to complain about political 
axes to grind.


Mike

On 4/27/21 10:50 PM, Mel Beckman wrote:
NANOG is not the right place to post this. This list is not an 
“interesting news group”, and as fascinating as the patent troll take 
down is, it has nothing to do with operational issues. Read the AUP, 
if your don’t believe me. Item 8:


Posts of a political, philosophical, or legal nature are prohibited.

I for one don’t want the list to be overrun again by people with a 
political axe to grind, no matter how noble.


 -mel

On Apr 27, 2021, at 3:34 PM, Justin Paine via NANOG  
wrote:



Correction -- another one. 
https://blog.cloudflare.com/winning-the-blackbird-battle/ 
<https://blog.cloudflare.com/winning-the-blackbird-battle/>  :)


Here's an except from the new blog post:

offering $100,000 to be shared by the winners who are successful in 
finding such prior art.


Please help!

__
*Justin Paine*
He/Him/His
Head of Trust & Safety
101 Townsend St, San Francisco, CA 94107

*PGP:* BBAA 6BCE 3305 7FD6 6452 7115 57B6 0114 DE0B 314D 
<https://keys.openpgp.org/vks/v1/by-fingerprint/BBAA6BCE33057FD66452711557B60114DE0B314D>




On Tue, Apr 27, 2021 at 3:26 PM Michael Thomas <mailto:m...@mtcc.com>> wrote:



And we can help! Cloudflare is setting out to destroy a patent troll:


https://www.techdirt.com/articles/20210426/09454946684/patent-troll-sable-networks-apparently-needs-to-learn-lesson-cloudflare-wants-to-destroy-another-troll

<https://www.techdirt.com/articles/20210426/09454946684/patent-troll-sable-networks-apparently-needs-to-learn-lesson-cloudflare-wants-to-destroy-another-troll>

Mike

Re: Something that should put a smile on everybody's face today

[Michael Thomas]

On 4/28/21 2:04 AM, William Herrin wrote:

On Tue, Apr 27, 2021 at 10:51 PM Mel Beckman  wrote:

NANOG is not the right place to post this. This list is not an “interesting 
news group”, and as fascinating as the patent troll take down is, it has 
nothing to do with operational issues. Read the AUP, if your don’t believe me. 
Item 8:

A major North American Operator goes after some industry boogeymen who
tried to extort them with a router (Networking) patent. Seems pretty
on topic to me.

Doubly so because this is exactly the right community that can help 
eliminate an industry scourge with its knowledge of prior art, etc.


Mike

Re: Something that should put a smile on everybody's face today

[Michael Thomas]

On 4/28/21 10:19 AM, Mel Beckman wrote:

Michael,

Sorry, but Cloudfare wasn’t sued /because/ they’re a service provider. 
This dispute is no different than if they had gotten into an argument 
over a copier toner scammer. And your snide remark about my comments, 
claiming they are political, is uncalled for.  I fastidiously avoid 
making political comments, and take pains to explain my operational 
concerns if there might be any doubt (as I did with the Parler 
cancellations).


I never said the copyright troll issue isn’t important. It just 
doesn’t belong on NANOG. It hinges entirely on philosophical issues 
with the PTO.


Snort. Your gubbermint conspiracy theories about the DoD address space 
dripped of politics.


They were sued because they are a service provider with money and they 
are fighting back asking for the community to help out. As William said, 
that seems pretty on-topic to me. This community is in a good position 
to provide that help which would be of benefit to NANOG in general. 
Again, on-topic for network operators.


Mike

Re: Google uploading your plain text passwords

[Michael Thomas]

[sorry meant to send this to the list]

Isn't that what lots of password managers do? I understand that one of them
syncs point to point, but that has the downside that it probably needs to
be on the same subnet.

The actual problem here is that sites only allow a single password. if you
could enroll more than one password you wouldn't need to sync at all.
Better: use asymmetric keys and enroll public keys so the secret never
leaves your device.

Mike

On Fri, Jun 11, 2021 at 9:53 AM William Herrin  wrote:

> On Fri, Jun 11, 2021 at 9:42 AM César de Tassis Filho
>  wrote:
> > Google does not have access to your plain-text passwords in either case.
>
> If they can display the plain text passwords to me on my screen in a
> non-Google web browser then they have access to my plain text
> passwords. Everything else is semantics.
>
> Regards,
> Bill Herrin
>
>
> --
> William Herrin
> b...@herrin.us
> https://bill.herrin.us/
>

Re: Google uploading your plain text passwords

[Michael Thomas]

On Fri, Jun 11, 2021 at 12:01 PM William Herrin  wrote:

> On Fri, Jun 11, 2021 at 10:27 AM Michael Thomas  wrote:
> > Isn't that what lots of password managers do? I understand that one of
> them syncs point to point, but that has the downside that it probably needs
> to be on the same subnet.
>
> It's exactly what lots of password managers with browser extensions
> do. I don't personally use them because I don't want my passwords
> reversibly stored on a computer that I don't directly control. I have
> no great philosophical problem with their existence and use by those
> who want them, I just don't want them for myself.
>

Well, browser extensions in and of themselves scare the living hell out of
me.  It really surprises me that they aren't a major attack vector and in
the news all of the time.

But yes, I agree that even encrypted they are a *very* tempting target for
hackers, and especially foreign governments. A breach would mean that
everybody is instantly screwed since they don't have to break into
individual computers, install malware, etc.

Mike

Can somebody explain these ransomwear attacks?

[Michael Thomas]

Not exactly network but maybe, but certainly operational. Shouldn't this 
just be handled like disaster recovery? I haven't looked into this much, 
but it sounds like the only way to stop it is to stop paying the crooks. 
There is also the obvious problem that if they got in, something (or 
someone) is compromised that needs to be cleaned which sounds sort of 
like DR again to me.


Mike

Re: Can somebody explain these ransomwear attacks?

[Michael Thomas]

On 6/24/21 2:55 PM, JoeSox wrote:


It gets tricky when 'your' company will lose money $$$ while you wait 
a month to restore from your cloud backups.
So Executives roll the dice to see if service can be restored quickly 
as possible keeping shareholders and customers happy as possible.


But if you pay without finding how they got in, they could turn around 
and do it again, or sell it on the dark web, right?


Mike




On Thu, Jun 24, 2021 at 2:44 PM Michael Thomas <mailto:m...@mtcc.com>> wrote:



Not exactly network but maybe, but certainly operational.
Shouldn't this
just be handled like disaster recovery? I haven't looked into this
much,
but it sounds like the only way to stop it is to stop paying the
crooks.
There is also the obvious problem that if they got in, something (or
someone) is compromised that needs to be cleaned which sounds sort of
like DR again to me.

Mike

Re: Can somebody explain these ransomwear attacks?

[Michael Thomas]

On 6/24/21 3:08 PM, Shane Ronan wrote:
A lot of the payments for Ransomware come from Insurance Companies 
under "Business Interruption Insurance". It in fact may be more cost 
effective to pay the ransom, than to pay for continued business 
interruption.


Of course along with paying the ransom, a full forensic audit of the 
systems/network is conducted. The vector for many of these attacks is 
via a worm triggered by someone opening an attachment on an email or 
downloading compromised software from the Internet. Short of not 
allowing email attachments or blocking Internet access, the best 
method is to properly train users to not click on attachments or visit 
"untrusted" sites, but nothing is perfect.



I wonder if this is preying off the firewall 
hard-on-the-outside-soft-on-the-inside? At this point I'm not sure how 
you can justify that because so many people are using their own 
equipment. It's not just the operational side of the business they can 
target, after all.


Mike

Re: OT: Re: Can somebody explain these ransomwear attacks?

[Michael Thomas]

On 6/24/21 4:57 PM, Karl Auer wrote:

Ransomwear - the latest fashion idea.

"Pay me money or I will continue to wear these clothes"

I reckon I could make a killing just by stepping out in a knee-length
macrame skirt...

Lol. Thanks, I knew that didn't look right. Maybe with a crop top to 
complete the ensemble.


Mike

Re: Can somebody explain these ransomwear attacks?

[Michael Thomas]

On 6/25/21 5:25 AM, Jim wrote:

On Thu, Jun 24, 2021 at 5:41 PM Brandon Svec via NANOG  wrote:

I think a big problem may be that the ransom is actually very cost effective 
and probably the lowest line item cost in many of these situations where large 
revenue streams are interrupted and time=money (and maybe also health or life).

Big problem that with organizations' existing Disaster Recovery DR methods --
the time and cost to recovery from any event including downtime will
be some amount.. likely a high one,
and criminals' ransom demands will presumably be set as high a price
as they think they can get --
but still orders of magnitudes less than cost to recover / repair /
restore, and the downtime may be less.

The  ransom price becomes the perceived cost of paying from the
perspective of the
organizations faced with the decision,  But the actual cost to the
whole world of them paying
a ransom is much higher and will be borne by others (And/or themselves
if they are unlucky)
in the future, when their having paid the criminals encourages and
causes more and more of that nefarious activity.


Well, the cost of the DR fire drill is proportionate to how automated, 
etc, it is. If you think that the odds of a DR event are really low you 
want to make it possible but not necessarily cheap. If it happens all of 
the time, you want to optimize for speed and efficiency.


The object here is to break their business model, at least for you. Even 
if you go through one DR they aren't likely to go back again rather than 
finding another sucker.


Mike

Re: Can somebody explain these ransomwear attacks?

[Michael Thomas]

On 6/25/21 8:39 AM, Karl Auer wrote:

On Fri, 2021-06-25 at 10:05 -0400, Tom Beecher wrote:

Everything can be broken, and nothing will ever be 100% secure. If
you strive to make sure the cost to break in is massively larger than
the value of what could be extracted, you'll generally be ahead of
the game.

Easy to say.

IMHO the only workable long-term defence is heterogeneity - supported
by distribution, redundancy and just taking the simple things
seriously.

Business has spent the last few decades discarding heterogeneity and
the bigger they are, the more comprehensively they have discarded it.
Companies that are floor to ceiling and wall to wall Windows.
Centralised updates, centralised networking, centralised storage,
centralised ops teams, and (typically) a culture of sharing. A
relentless prioritising of convenience over security. For goodness
sake, even the NSA had the attitude that "if you are this side of the
drawbridge you must be OK"!

We need to start building systems that are not seamless, that are not
highly interchangeable, that are not fully interconnected, and we have
to include our human systems in that approach.

How does one go about that in real life? You certainly want your servers 
patched with the latest security updates. For all intents and purposes 
there is just Windows and Linux. I suppose you could throw in some 
hardware diversity with ARM or MIPS.


Routers are definitely in better shape on that front as there are lots 
of choices and at least Cisco has tons of different BU's that compete 
with each other with different software and hardware.


Mike

Re: Can somebody explain these ransomwear attacks?

[Michael Thomas]

On 6/25/21 11:59 PM, Valdis Klētnieks wrote:

On Thu, 24 Jun 2021 14:55:12 -0700, JoeSox said:


It gets tricky when 'your' company will lose money $$$ while you wait a
month to restore from your cloud backups.

If that's a concern, you've *already* totally screwed the pooch regarding DR 
planning.

So what is the industry standard if there is one for DR recovery? 
Shouldn't this just be considered another hit by the Chaos Monkey?


Mike

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[Michael Thomas]

On 6/30/21 11:30 AM, Sean Donelan wrote:


STIR/SHAKEN Broadly Implemented Starting Today
https://www.fcc.gov/document/stirshaken-broadly-implemented-starting-today 



WASHINGTON, June 30, 2021—FCC Acting Chairwoman Jessica Rosenworcel today
announced that the largest voice service providers are now using 
STIR/SHAKEN caller ID authentication standards in their IP networks, 
in accordance with the deadline set by the FCC. This widespread 
implementation helps protect consumers against malicious spoofed 
robocalls and helps law enforcement track bad actors. The STIR/SHAKEN 
standards serve as a common digital language used by phone networks, 
allowing valid information to pass from provider to provider which, 
among other things, informs blocking tools of possible suspicious calls.



Just because you can know (fsvo "know") that a call is allowed to assert 
a number doesn't change anything unless other actions are taken. With 
DKIM which is far simpler than STIR it would require reputation systems 
that don't seem to have been deployed, submission auth which thankfully 
was deployed, policy enforcement (ie ADSP) which is not deployed, and 
user indicators which are sporadically deployed.


Given the giant security holes caused by solving the wrong problem (ie 
trying to authenticate the e.164 address rather than the originating 
domain) it's just going to push spammers to exploit those holes. It's 
very much to be seen whether victory can be declared, IMO.


Mike

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[Michael Thomas]

On 6/30/21 12:17 PM, Paul Timmins wrote:


On 6/30/21 2:56 PM, Michael Thomas wrote:
Just because you can know (fsvo "know") that a call is allowed to 
assert a number doesn't change anything unless other actions are 
taken. With DKIM which is far simpler than STIR it would require 
reputation systems that don't seem to have been deployed, submission 
auth which thankfully was deployed, policy enforcement (ie ADSP) 
which is not deployed, and user indicators which are sporadically 
deployed.


In any indication, the carrier closest to the originator is signing 
the call metadata with their digital certificate. While this won't 
mean much to the active user, for those tracking down robocalls, this 
is the holy grail - finding the carrier who is letting the calls into 
the network and being able to reach out to them to stop the 
abusive/illegal traffic.


As I said, STIR solved the wrong problem. I know domains as a user. I 
have no clue about e.164 address ranges. Also: this is 2021 and e.164 
address need to go the way of the dodo.


From an automated standpoint, I really don't care about whether a phone 
number is authentic, I care about the domain that onramped it so I can 
theoretically punish it. It's the people who are allowing the spoofing 
that is the real problem which directly analogous to email open relays.


Also: reputation is nice in theory but I am dubious that it is deployed 
in reality. Given the entire ARC farce which was driven by Google -- who 
owns gmail -- to supposedly "solve" the mailing list traversal problem 
but boils down to a reputation system, that strongly suggests that they 
don't have one either. I'm not sure why we should be optimistic about 
that for STIR which solves for a much harder problem which is inherently 
not entirely secure given SS7 gateways.





That it might say we've taken the time to verify the end user is who 
they say they are is just icing on the cake. The goal is to make the 
calls accountable to someone, which despite the patchwork of systems 
in the US that might prevent the signature from coming through, can 
help a lot since the biggest wholesalers have implemented it 
(Inteliquent and Lumen among many others)


The other big deal is that now all carriers are actually expected to 
police their network for spoofed callers who are exhibiting 
robocalling behavior. This is a big deal! For the first time, carriers 
are going to be held responsible for proactively finding the abuse, 
and showing what their plans are to do such a thing, and sharing 
information with each other (via the FCC) who can be contacted to 
chase down robocall traffic if another carrier sees it.


I'm not trying to say that it's not a good thing to have authentication, 
but as implemented by STIR it's ridiculously more complex than it needed 
to be had they chosen the right problem to solve which is to know the 
domain that is onramping the call. This could have been trivially rolled 
out a decade ago and I even experimented DKIM signing SIP message about 
15 years ago.


It's never been entirely clear whether DKIM was the impetus for cleaning 
up open relays. I'd like to think it was, but the more likely 
explanation was that it was in the water at the time. The FCC could have 
at any time just clamped down on that from a regulatory standpoint 
without going to all of the rigamarole of STIR. Email doesn't have a 
similar regulatory body to lean on so we had to take it into our own hands.





Given the giant security holes caused by solving the wrong problem 
(ie trying to authenticate the e.164 address rather than the 
originating domain) it's just going to push spammers to exploit those 
holes. It's very much to be seen whether victory can be declared, IMO.


Fortunately, positive identification of the caller isn't the intent. 
Preventing people from pretending to be the IRS is the intent.



e.164 addresses don't allow me to know if something is from the IRS. 
irs.gov does. Also, papers have shown that UI identification is a net 
positive which is a shame given how sporadically they are done and how 
inconsistent the UI's are. If they were widespread it would probably 
much better.


Mike

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[Michael Thomas]

On 7/1/21 1:05 PM, Paul Timmins wrote:



On 7/1/21 3:53 PM, Keith Medcalf wrote:

And this is why this problem will not be solved.  The "open relay" is making money from processing 
the calls, and the end carrier is making money for terminating them.  Until fine(s) -- hopefully millions of 
them, one for each improperly terminated call, together with jail time for the CEO of the company for 
"conspiracy to commit fraud" --  and EACH of the fines is EQUAL OR GREATER than the total yearly 
worldwide REVENUE of that end carrier, they will not have any impetus to "fix" the problem.


How about 47 CFR 64.1200(k)(4)?

(4) A provider may block voice calls or cease to accept traffic from 
an originating or intermediate provider 
 
without liability under the Communications Act or the Commission's 
rules where the originating or intermediate provider 
, 
when notified by the Commission, fails to effectively mitigate illegal 
traffic within 48 hours or fails to implement effective measures to 
prevent new and renewing customers 
 
from using its network to originate illegal calls. Prior to initiating 
blocking, the provider shall provide the Commission with notice and a 
brief summary of the basis for its determination that the originating 
or intermediate provider 
 
meets one or more of these two conditions for blocking.


ie: "You're not really a phone company anymore, says the rest of the PSTN"

https://www.law.cornell.edu/cfr/text/47/64.1200

Those who fail to understand the Usenet Death Penalty are doomed to 
(not) repeat it.


Mike

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[Michael Thomas]

People who are actually interested in this subject are well advised to 
read this thoroughly because it equally applies to SIP spam with a 
system far less complex and far fewer gaping security holes as STIR.


https://www.usenix.org/system/files/conference/usenixsecurity18/sec18-hu.pdf

Mike

On 7/2/21 8:54 AM, Paul Timmins wrote:


Fun part is that just because it's a telnyx number with a checkmark, 
it doesn't mean the call came from Telnyx, just that the call came 
from a carrier that gave the call attestation A. As the carrier, we 
can see who signed the call (it's an x509 certificate, signed by the 
STI-PA, with the carrier's name and OCN in it) and hold them 
accountable for the traffic, which is huge.


But that's where the confusion will lie - a customer might say well 
this is a verizon wireless number, i'll yell at them! But the actual 
call came in through Lumen, and they're the ones who can stop it. A 
carrier can see the cert, but you can just get the verstat flag from 
the P-Asserted-Identity field in the call to the handset and see that 
it passed the tests for attestation A.


Just because you don't see a checkmark doesn't mean signatures aren't 
happening. Attestation B and C aren't displayed on the handset (but 
are seen in the carrier's systems) and most androids don't have a way 
to display stir/shaken stuff yet. T-Mobile doesn't send the verstat 
header to handsets they don't verify as s/s compliant (usually only 
ones they sell). My trick was to sim swap into an iphone for a day, 
then back to my android which started displaying the verification 
after that.


It's all new, but just because you don't see it doesn't mean it's not 
there. Report the calls to your carrier, they have new tools to track 
down the misbehavior.


On 7/2/21 8:32 AM, Nick Olsen wrote:
Not all have implemented it yet. But if you haven't. You were 
supposed to implement some kind of robo calling mitigation plan (Or 
atleast certify that you have one). At $dayjob we're fully deployed 
(inbound and outbound).


I received my first ever STIR/SHAKEN signed (iPhone Check mark, 
highly scientific) spam call on my personal Cell phone on 6/30. It 
was a Telnyx number. Had the call terminated to $dayjob network. I 
fully would have collected all various information and ticketed it 
with Telnyx.


Time will tell how truly effective this is. But we have better 
originating information now (breadcrumbs) to follow back to the source.


On Thu, Jul 1, 2021 at 5:42 PM Andreas Ott > wrote:




On Thu, Jul 1, 2021 at 12:56 PM Keith Medcalf
mailto:kmedc...@dessus.com>> wrote:

... and the end carrier is making money for terminating them. 



Survey (of n=1) says: nothing has changed, aka the new technology
is not working. I just received the same kind of recorded message
call of "something something renew auto warranty" on my AT&T
u-Verse line. This time when I called back the displayed caller
ID number it was ring-no-answer, versus the previous "you have
reached a number that is no longer in service". By terminating
the call the carrier made probably more money than it would cost
them to enforce the new rules.

Other than the donotcall.gov  portal, is
there a new way to report the obvious failure of STIR/SHAKEN?

-andreas

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[Michael Thomas]

Nothing has changed for me either. Color me surprised. The real proof 
will be to see if the originating domain can be determined, and whether 
the receiving domain does anything about it.


Mike

On 7/9/21 9:42 AM, Brandon Svec via NANOG wrote:
I’m getting the same or more, but did anyone really expect they would 
stop July 1? It will take time for complaints to be tracked down and 
operators to take actions, right?


Brandon

On Fri, Jul 9, 2021 at 6:49 AM Josh Luthman 
mailto:j...@imaginenetworksllc.com>> wrote:


Subjectively speaking, I'm still getting the same amount of spam
phone calls.

I'm certainly getting a lot more spam SMS to my cell.  Almost all
of them in my entire life starting July 1...


Josh Luthman
24/7 Help Desk: 937-552-2340
Direct: 937-552-2343
1100 Wayne St


Suite 1337


Troy, OH 45373




On Fri, Jul 9, 2021 at 9:40 AM Jeff Shultz mailto:jeffshu...@sctcweb.com>> wrote:

All I know is that I am getting a lot fewer bogus calls on my
cell phone than I was this time last month.

On Fri, Jul 9, 2021, 06:17 Ryan Finnesey via NANOG
mailto:nanog@nanog.org>> wrote:

This should help with Robo calls a lot.

-Original Message-
From: NANOG
mailto:conovence@nanog.org>> On Behalf Of Sean Donelan
Sent: Wednesday, June 30, 2021 2:31 PM
To: nanog@nanog.org 
Subject: SITR/SHAKEN implementation in effect today (June
30 2021)


STIR/SHAKEN Broadly Implemented Starting Today

https://www.fcc.gov/document/stirshaken-broadly-implemented-starting-today



WASHINGTON, June 30, 2021—FCC Acting Chairwoman Jessica
Rosenworcel today announced that the largest voice service
providers are now using STIR/SHAKEN caller ID
authentication standards in their IP networks, in
accordance with the deadline set by the FCC. This
widespread implementation helps protect consumers against
malicious spoofed robocalls and helps law enforcement
track bad actors. The STIR/SHAKEN standards serve as a
common digital language used by phone networks, allowing
valid information to pass from provider to provider which,
among other things, informs blocking tools of possible
suspicious calls.

--
Brandon Svec
15106862204 ☎️ or 💬

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[Michael Thomas]

On 7/9/21 1:36 PM, K. Scott Helms wrote:
Nothing will change immediately.  Having said that, I do expect that 
we will see much more effective enforcement. The investigations will 
come from the ITG (Industry Traceback Group) with enforcement 
coming from FCC or FTC depending on the actual offense.  The problem 
has been that it's been far too easy for robocalling companies to hop 
from one telecom provider to another.  Now there are requirements 
around "know your customer" that telecom operators have to follow and 
the ITG will have a much better chance of figuring out who the bad 
actor is than they have in the past.


The thing is that that shouldn't have been held up by rolling out STIR. 
With email, there was nothing akin to the FCC so it was really the only 
name-and-shame stick we had. This could have been done years ago.





Longer term I worry that this will lead to more attacks on PBXs, 
eSBCs, and VOIP handsets to be able to call either from that endpoint 
itself or be able to use the SIP credentials. The market for robocalls 
will certainly not disappear.


A meta question that really needs to be asked these days is why we even 
need telco telephony anymore. A lot of problems go away if you are not 
in thrall to 100 year old technology and its accreted kruft.


Mike

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[Michael Thomas]

On 7/9/21 3:32 PM, K. Scott Helms wrote:




On Fri, Jul 9, 2021 at 4:47 PM Michael Thomas <mailto:m...@mtcc.com>> wrote:



On 7/9/21 1:36 PM, K. Scott Helms wrote:
> Nothing will change immediately.  Having said that, I do expect
that
> we will see much more effective enforcement. The investigations
will
> come from the ITG (Industry Traceback Group) with enforcement
> coming from FCC or FTC depending on the actual offense.  The
problem
> has been that it's been far too easy for robocalling companies
to hop
> from one telecom provider to another.  Now there are requirements
> around "know your customer" that telecom operators have to
follow and
> the ITG will have a much better chance of figuring out who the bad
> actor is than they have in the past.

The thing is that that shouldn't have been held up by rolling out
STIR.
With email, there was nothing akin to the FCC so it was really the
only
name-and-shame stick we had. This could have been done years ago.


It wouldn't work the same and I say that as someone who ran email for 
ISPs for decades and just got done with a STIR/SHAKEN implementation.  
There's far more money in robocalls than there ever has been in spam.  
The other thing is that the technologies are fundamentally different.  
Don't get me wrong, there are parallels, but comparing email to the 
various flavors of telephony (POTS, SIP, MGCP, H.323, etc) isn't all 
that useful because they're so different. When I get an email as a 
provider I can always figure out the path it took to get to me.  The 
same is not at all true for a call, though I can trace it to a degree 
via the CDRs from carriers I work with.  Much of the call path will be 
opaque and often in the case of robocallers it's intentionally so.  
Number porting is another (big) difference.  We could always choose to 
forward email for a customer who left our service, but imagine if 
email literally let that person take their address with them 
regardless of who was providing the hosting for the email.


Once it hits a SIP gateway it's pretty much the same. The thing that I 
think that made the made the biggest -- and this coming from one of the 
inventors of DKIM -- was shutting down open relays. With email there was 
no back pressure on ESP's to close them so DKIM was at least a way to 
name and shame, or at least that was one of the goals at least in my 
mind. It's hard to say whether there was actually cause and effect, but 
the reality now is that open relays are pretty much gone -- at least 
with ESP's.


I was one of the early Cassandras telling people that 
P-Asserted-Identity was going to lead to exactly what has happened for 
which I got told I was wrong, and then threw up my hands and went and 
worked on email. This could have been dealt with 15 years ago and it 
could have trivially piggybacked off of the DKIM work -- heck I even 
hacked a SIP stack to prove the point. And it should have learned the 
lessons with email which it apparently has not because the 9th of July 
don't seem any different than the 30th of June.


Also: since there are PSTN gateways which fundamentally can't be secured 
spammers will take advantage of the holes as they become economically 
viable. The entire scheme of attesting for e.164 addresses was a 
complete waste of time. The problem is with the SIP gateway that onramps 
the bogus calls not whether somebody should be able to assert a given 
address real time. Those onramps could have taken those measures a 
decade ago but didn't just like the open email sewers before requiring 
submission auth. I think that Peterson has some other wacky shit to make 
PSTN gateway traversal better, turning a Rube Goldberg contraption into 
something even worse. The entire thing is madness with its complexity 
but I would expect nothing less from the SIP WG.




> Longer term I worry that this will lead to more attacks on PBXs,
> eSBCs, and VOIP handsets to be able to call either from that
endpoint
> itself or be able to use the SIP credentials. The market for
robocalls
> will certainly not disappear.
>
A meta question that really needs to be asked these days is why we
even
need telco telephony anymore. A lot of problems go away if you are
not
in thrall to 100 year old technology and its accreted kruft.


Robocalls really aren't a product of the legacy PSTN. Today almost 
none of them originate from anywhere but VOIP. Now, you can certainly 
say that if SS7 had robust authentication mechanisms that we could 
then trust caller ID (more) but there's no sign of us abandoning the 
PSTN anytime soon.  Having said that, there's any number of protocols 
we rely on today that have the exact same gap.  BGP is arguably even 
worse than SS7.

Re: SITR/SHAKEN implementation in effect today (June 30 2021)

[Michael Thomas]

On 7/9/21 3:44 PM, Keith Medcalf wrote:

On Friday, 9 July, 2021 16:32, K. Scott Helms wrote:
Robocalls really aren't a product of the legacy PSTN.  Today almost none
of them originate from anywhere but VOIP.  Now, you can certainly say
that if SS7 had robust authentication mechanisms that we could then trust
caller ID (more) but there's no sign of us abandoning the PSTN anytime
soon.  Having said that, there's any number of protocols we rely on today
that have the exact same gap.  BGP is arguably even worse than SS7.

The root of the problem is that the "Caller ID" is not a "Caller ID".  If there were a requirement 
for "truth in advertizing" it would properly be called the "Caller Advertizement" because it is 
primarily intended as an advertizement by the caller, and not an ID of the caller.

The assumption back around 2004 was that P-Asserted-ID would be an old 
boys network to the end and get told to buzz off when I complained that 
it wouldn't. This was trivially foreseeable  back then and it played it 
the most ridiculously foreseeable way.


Mike