Hi Joel,
Full disclosure: I’m a crypto and web application guy. I have some
exposure to DNS and TCP, but I had to google “BGP” while reading this
draft. I tried my best, but I probably don’t have even the minimum
required background knowledge to be security reviewing this draft.
My (limited) understanding of DNS Resolvers is that if you bridge
networks such that the foreign network has access to your network’s
DNS resolvers, then agents on the foreign network can now query any
FQDN and find out if it exists on your network and maybe deduce some
topological information based on IP address (aka, “map your network”).
I could well imagine that in the type of net2cloud setups you are
describing, you would want the cloud to be able to resolve certain
workloads on the on-prem network (ex.: build agent worker nodes), but
not necessarily everything on your network (ex.: laptops and
workstations). I tried to provide the example that just knowing that,
for example, all the workstations under the domain “corp.acme.com”
suddenly become resolvable on your corporate network might indicate a
merger & acquisition that is not public knowledge yet. To my
understanding, DNS is not really designed for privacy, ie to hide the
existence of certain subnets depending on who the requester is (unless
you do some super complicated recursive DNS setups). Maybe Linda is
right and this is out-of-scope for this document, but it feels to me
like some discussion of this point would fit into section 3.5.
Similarly, my understanding of NATs is that they function by having a
gateway proxy traffic from its subnet to a wider area network (or vice
versa from a wide area network into a subnet), such that it appears to
firewalls and other traffic monitoring software as if that traffic
originates from the gateway itself. Being overly cavalier about using
NATs to bridge trusted and untrusted networks feels like it would
really hamper security traffic monitoring tools and firewalls from
being able to properly enforce origin / destination based rules. Maybe
Linda is right and this is out-of-scope for this document, but it
feels to me like some discussion of this point would fit into section 3.6.
I am far less familiar with the other technologies mentioned in the
rest of sections 3, 4, and 5, but I would assume that anywhere that
you’re bridging a trusted network (aka a network you own) to an
untrusted network (aka a network you don’t own), you want to be a bit
careful about what suddenly now has access to what. In my $dayjob, we
often perform security modelling exercises where we look at a network
component, color it red and say “Ok, that’s compromised, what can it
see? Now color all of those red”. Granted, as Linda said, we are
largely thinking about application layer here, but still,
vulnerabilities and exploitable configurations can and do exist at all
layers and in this regard, subnet boundaries are extremely important
for containing the spread of bad things and relaxing subnet boundaries
should be done with extreme caution. In that this is a problem
statement document, I’m sure there is something you can say in each
section to this point.
---
*Mike*Ounsworth
*From:*Joel Halpern <j...@joelhalpern.com>
*Sent:* Wednesday, September 11, 2024 7:14 PM
*To:* Mike Ounsworth <mike.ounswo...@entrust.com>; sec...@ietf.org;
Deb Cooley <debcool...@gmail.com>; Tero Kivinen <kivi...@iki.fi>
*Cc:* draft-ietf-rtgwg-net2cloud-problem-statement....@ietf.org;
rtgwg@ietf.org
*Subject:* [EXTERNAL] Re: Seder early review of
draft-ietf-rtgwg-net2cloud-problem-statement-41
I know and respect that you asked that a different reviewer be
appointed. However, as Shepherd I am trying to figure out if there is
an underlying problem that Linda and I were not able to get at from
reading your review. The document is about
I know and respect that you asked that a different reviewer be
appointed. However, as Shepherd I am trying to figure out if there is
an underlying problem that Linda and I were not able to get at from
reading your review.
The document is about the problems edge devices encounter
inclassifying traffic that they are handling to and from data centers
(thus net2cloud). Apparently, from the comments I think I understood,
you are expecting fine grained application level differentiation for
security purposes to be part of the problems this draft addresses.
1) Did I misunderstand what you expected / wanted / sought?
2) Can you help us understand what wording in the document led to that
expectation, so we can clarify the document?
Thank you,
Joel Halpern (document Shepherd)
On 9/11/2024 11:56 AM, Mike Ounsworth wrote:
Hi Linda,
Alright, you’ve rejected every one of my review comments as being
out-of-scope. I clearly lack the domain knowledge to review this
document properly. @Deb Cooley <mailto:debcool...@gmail.com> or
@Tero Kivinen <mailto:kivi...@iki.fi> can you please re-assign
this SecDir review to a different reviewer who has the appropriate
Routing domain expertise?
---
*Mike*Ounsworth
*From:*Linda Dunbar <linda.dun...@futurewei.com>
<mailto:linda.dun...@futurewei.com>
*Sent:* Tuesday, September 10, 2024 2:53 PM
*To:* Mike Ounsworth <mike.ounswo...@entrust.com>
<mailto:mike.ounswo...@entrust.com>; sec...@ietf.org
*Cc:* draft-ietf-rtgwg-net2cloud-problem-statement....@ietf.org;
rtgwg@ietf.org
*Subject:* [EXTERNAL] RE: Seder early review of
draft-ietf-rtgwg-net2cloud-problem-statement-41
Mike, Thank you very much for the review and the comments. Please
see below the resolutions to your comments and suggestions. Linda
-----Original Message----- From: Mike Ounsworth via Datatracker
<noreply@ ietf. org> Sent: Tuesday, September
Mike,
Thank you very much for the review and the comments.
Please see below the resolutions to your comments and suggestions.
Linda
-----Original Message-----
From: Mike Ounsworth via Datatracker <nore...@ietf.org>
Sent: Tuesday, September 10, 2024 7:11 AM
To: sec...@ietf.org
Cc: draft-ietf-rtgwg-net2cloud-problem-statement....@ietf.org;
rtgwg@ietf.org
Subject: Secdir early review of
draft-ietf-rtgwg-net2cloud-problem-statement-41
Reviewer: Mike Ounsworth
Review result: Has Issues
I have reviewed this document as part of the security
directorate's ongoing effort to review all IETF documents being
processed by the IESG. These comments were written primarily for
the benefit of the security area directors. Document editors and
WG chairs should treat these comments just like any other last
call comments.
The summary of the review is that while the Security
Considerations section lists a few specific good points, it does
not address the more fundamental issue that when you bridge a
network you own to a network that you don't, you should consider
at ever level how much and how deep you're intending that access
into your network to be. -- For example: I may want O365 to see my
on-prem Exchange server, but not to be able to build a network map
of all corporate laptops and workstations.
[Linda] O365 is application level, which is out of the scope of
this document. This document identifies high-level problems that
can be addressed by protocol extensions within the IETF routing
area domain. Other Cloud DC problems (e.g., managing Cloud
spending) are out of the scope of this document.
This review is broken up into "Security Comments" first, and then
"Editorial Comments".
Security Comments
For some levity, I will start my security review with a comic that
I think brilliantly summarizes the situation
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fxkcd.com%2F2044%2F&data=05%7C02%7Clinda.dunbar%40futurewei.com%7C91333bfec45a4c29563b08dcd1a25a85%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C638615742483976543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C0%7C%7C%7C&sdata=2a3pwixOSdvzVMTh62K%2Bx%2F5vp8%2B8lIIat2ObG5E6Y5E%3D&reserved=0
<https://urldefense.com/v3/__https:/nam11.safelinks.protection.outlook.com/?url=https*3A*2F*2Fxkcd.com*2F2044*2F&data=05*7C02*7Clinda.dunbar*40futurewei.com*7C91333bfec45a4c29563b08dcd1a25a85*7C0fee8ff2a3b240189c753a1d5591fedc*7C1*7C0*7C638615742483976543*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C0*7C*7C*7C&sdata=2a3pwixOSdvzVMTh62K*2Bx*2F5vp8*2B8lIIat2ObG5E6Y5E*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUlJSUlJSUl!!FJ-Y8qCqXTj2!e-q0IrunoecSiqEVNkzb0uuZ3oecLCoBiDnC84-iaEDIDKKAK8i8WjOs_jQARHzhWRy5OeKz-URJo6fbgD5ILHQ-hkDSiA$>
This draft is hitting the "I wish these parts could communicate
more easily", "Integrate Everything!" boxes, and it's addressing
some of the "Oh-oh, there are so many connections. It's creating
Bugs and Security Holes" box, but it's not addressing the final
box that often (especially for security) segmentation and
sandboxing is a good thing, and you need to be mindful of keeping
that segmentation where it matters.
[Linda] Thank you very much for sharing your brilliant comic. It
is very interesting. But those sandboxes are out of the scope of
IETF Routing Area.
Secton 3
This document would benefit from some discussion of thinking about
where your "sensitive" applications and data is, and what needs to
be protected from what.
[Linda] That would need another document in IETF Application Area
to describe “sensitive” applications and data placement in Cloud.
The Net2Cloud document is to describe high-level problems that the
IETF Routing area can address.
Absent is discussion that sometimes the drive to multi-cloud or
hybrid-cloud is actually motivated by security. Ex1: sometimes you
have to run super risky workloads (like user-submitted code or
virus samples), and you know that cloud providers can build a
better sandbox than you can, so the motivation is to get the risky
stuff off your on-prem network. Ex2: You may put a highly
sensitive server in the cloud for its own protection so that it is
isolated from a potential compromise of your on-prem network. Ex3:
sometimes you want to keep your sensitive data (ex.: financial,
contracts, pre-patent products, etc) on-prem so that you can have
extremely tight control of it. Depending on whether you're trying
to protect on-prem from cloud or protect cloud from on-prem, this
leads to different design considerations across pretty much every
section of this document.
[Linda] All those problems you have identified are valid and need
guidelines. But they are not within the IETF Routing Area.
Probably consider writing a document in IETF Ops Area to address
issues and solutions for workloads in hybrid Clouds.
Section 3.1:
TL;DR: I am not an expert on GBP, but it very much feels like
there should be at least some relevant security considerations in
this section.
[Linda] The security considerations for network, like using
IPsec/MPLS, to connect to Cloud GW are in Section 7.
I am far from an expert in BGP, but my quick google shows that BGP
Peering is traditionally symmetric; ie the goal is to fully bridge
the two networks. In an On-Prem -> Cloud setup, you probably want
to consider separately whether you want to grant on-prem things
access to the cloud network vs granting cloud things access to the
on-prem network. You also want to consider whether you're
intending to grant access to *the entire* on-prem network, or only
certain subnets of it. This becomes a security issue pretty
quickly if you unintentionally grant workloads in a public cloud
with broad access to your on-prem network. This falls inline with
the sentence that is already in the
draft: "As such, there is pressure to peer more widely with more
customers, including those who may lack the expertise and
experience in running complex BGP peering relationships. This can
contribute to ..." because doing partial bridging / network
exposure is certainly more complex. It also begs the question of
whether the public cloud operators are properly isolating
BGP-peered customers from each other.
[Linda] Yes, isolating BGP-peered customers can be achieved by
VPN, BGP Communities for Traffic Engineering and Filtering. For
example, Azure allows the use of BGP communities to define the
scope of route advertisements in their peering configuration.
Similarly, AWS Direct Connect lets you tag routes with BGP
communities to control their propagation.
The draft-ietf-rtgwg-net2cloud-problem-statement- is NOT intended
to describe all the existing procedures for BGP peering. But to
identify some issues that can be solved by protocol extensions
within the IETF routing area domain.
Secton 3.5
Similar to my comment for 3.1: the described scenario is about
using your on-prem DNS servers to bridge two cloud DNS domains,
and sorta implied is that cloud-based workloads may need to
resolve on-prem resources. Absent is a discussion that you may not
want cloud-based workloads to be able to resolve (from a security
perspective, "map") your entire on-prem network; for example, even
if sensitive on-prem resources are not reachable due to TCP/UDP
firewalling, the ability for an attacker to build a network map of
hostnames and addresses may already be compromising in some
situations for example addition of certain hostnames / domains to
a local network may indicate a not-yet-publicly-announced merger,
acquisition, department, partnership, or new product feature. That
makes DNS Practices for Hybrid Workloads even more complex if you
only want to expose *part* of your on-prem DNS space to the cloud
workloads.
[Linda] the problem you described here are application layer
problem. Not within the scope of this document.
There should be some discussion of confidentiality: whether it's
an on-prem component resolving an cloud component, or vice-versa,
these are private internal components, and the DNS queries should
be considered sensitive metadata. So cross-network DNS queries
should be paired with an encryption layer such as DNS-over-HTTPS,
or ensuring that DNS queries are routed over a site-to-cloud VPN.
Section 3.6
Surely we could think up at least half a dozen security
considerations relating to overly-broad NATs punching
inintentional holes in firewalls. This is the section where the
XKCD comic really applies.
[Linda] interesting problem. But out of the scope of this document.
Are you sure you want your cloud workloads to have outbound to the
internet?
That enables compromised workloads to "phone home" to the
attacker's command-and-control server or reverse-shell server, or
to exfiltrate data to the attackers server, or .., or .., or.
Typically on-prem enterprises have Data Loss Prevention (DLP)
tools to detect this sort of behaviour leaving their network, but
public clouds may or may not have enterprise-grade DLP solutions
that monitor outbound connections from workloads for suspicious
traffic patterns.
We could also ask about whether compromised cloud workloads should
have access to resources outside their local subnet. In security
modelling, we typically consider subnets of workloads to be nice
security containers ... at least a compromise can't spread beyond
a subnet boundary; but if you start using NAT to punch holes in
your firewalls, then you don't have firewalls anymore and all bets
are off.
We could also ask about whether it's a good thing for things
outside a VPC to be able to reach things inside. Typically a
multi-tiered cloud application will bury sensitive things like
databases, HSMs, secrets vaults, legacy applications that don't
meet modern security requirements, etc deep in a backend VPC for
their own protection. Again, if you start using NAT to punch holes
in your firewalls, then you don't have firewalls anymore.
... speaking of the word "firewall", it only appears once in the
entire document (in the Introduction). I would think that this
word would feature prominently throughout pretty much every
section of a document that is fundamentally about bridging
networks of things you own and things you don't.
[Linda] very interesting problem. Which IETF area do you think we
can propose solutions?
Secton 3.7
This also has security concerns in that inbound tunnels to on-prem
networks often want to have IP based firewalling, or increasingly
AI-based anomaly detection, but constantly-shifting cloud sides of
the tunnel make this sort of thing hard.
[Linda] interesting problem. But out of the scope of this document.
Sections 4 & 5
Nothing new that isn't already mentioned above.
Editorial Comments
(I am a complete outsider to the routing space, so feel free to
disregard editorial comments if these points would be obvious to
the target audience of this document).
1. Introduction
Is it correct to interpret from the last sentence that the target
audience of this document is network admins for existing
enterprise networks who are considering adopting some form of
hybrid cloud? If so, this point becomes muddier further on as some
of the proposed mitigations seems like they need to be implemented
by the the public cloud infrastructure. Possibly the document
could benefit from more clearly separating "on-prem components",
"cloud components under the control of the customer" and "cloud
infrastructure under the control of the cloud operator", and being
clearer about where each suggested mitigation applies? As a
concrete example of this, Section 3.3 has
the sentence: " One method to mitigate the problems listed above
is to use
anycast
[RFC4786] for the services so that network proximity and conditions
can be automatically considered in optimal path selection."
which, at least to my layman's reading, is not clear whether this
is something the customer can do, or needs to be done by the cloud
operator. And then the other two suggested mitigations in that
section really sound like things the cloud operator need to implement.
[Linda] The intent of this document is to identify issues that can
be addressed by routing protocol extensions. So, the audience
should be IETF Routing Area participants.
Section 2:
Is it possible to expand the acronym for "SD-WAN"? It's used
several times throughout the document but never defined. I assume
it's "Software-Defined Wide Area Network"?
[Linda] You are correct. SD-WAN is defined in Section 2.
The term "BGP peering" is first used in 3.1, but I am unfamiliar
with it, and it was not defined in 2, it probably should be.
[Linda] BGP is the cornerstone of routing and a fundamental
protocol within the IETF community. BGP peering is well known.
Section 3.2
Term "IGP" used without any explanation or reference.
[Linda] IGP is another base routing protocol that is well known.
Term "EVPN" used without any explanation or reference.
[Linda] as stated in the document, RFC7432 describes the details
of EVPN.
Section 3.3
This section uses the terms "DNS Server", "client", "DNS
resolver", and "Local DNS resolver". It might enhance readabilty
if the text was explicit about which things are part of the
on-prem network (and therefor can be mitigated by on-prem
solutions), and which are part of the cloud.
[Linda] All those terms can be either an on-prem network or the cloud.
Secton 3.6
I had to read this sentence 3 times to parse it:
> By
configuration, some private subnets can have NAT functionality to
reach out to external networks, and some private subnets are
internal to a Cloud DC only.
I think a simple word change would help:
> By
configuration, some private subnets can have NAT functionality to
reach out to external networks, *WHILE* some private subnets are
internal to a Cloud DC only.
[Linda] Thanks for the suggestion. Changed in -v42 per your
suggestion.
Section 5.1
> in [Section 3] (Security Considerations)
The link "[Section 3]" does not appear to go to the Security
Considerations?
[Linda] Since this document is in the Routing Area, only one
section describe the Security Consideration for the potential
routing protocol extensions.
Linda
