Joel,
Thank you very much for attending the zoom call with Google's Ashok and
Oracle's Kausik to clarify the detailed requirements for Include - Regions for
Cloud Backbone Transit. The More accurate name should be Restricted Region List.
Here is the revised section for the Restricted Region. If it is Okay with you,
we will add it back to -06:
Linda
--------------------------------
4.5 Restricted Regions Sub-TLV
Some enterprises may require that traffic across the Cloud Backbone is strictly
confined to a specific set of regions. This Sub-TLV allows the ingress SD-WAN
CPE to express such restrictions as part of the encapsulation metadata.
Traffic MUST be discarded if the Ingress Gateway, the Egress Gateway, or any
transit node belongs to a region not listed in this Sub-TLV.
This restriction is commonly used to enforce regulatory, security, or
latency-based geographic constraints, where data must remain confined to
specified regions.
Format of the Restricted Regions Sub-TLV:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Type (=4) | Length | Reserved (16 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Region Len | Region ID or UTF-8 encoding of Region Name |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Region Len | Region ID or UTF-8 encoding of Region Name |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
- Type(8 bits): Assigned value identifying the Restricted Regions Sub-TLV.
Suggested value: 4.
- Length (8 bits): Total length of the Value field (everything after the Type
and Length fields), in octets.
- Reserved (16 bits): Reserved for future use. MUST be set to zero and
ignored on receipt.
- Region Len (8 bits per region entry): Length of the region identifier or
UTF-8 name, in octets.
- Region ID or UTF-8 encoding of the Region Name: Variable-length region
identifier. Can be a numeric Region ID or a UTF-8-encoded string (e.g.,
"us-west", "eu-central").
Multiple regions MAY be present, each starting with its own Region Len field.
Processing notes:
- Receiving Cloud Gateway MUST check whether it and all intermediate transit
regions are included in the listed regions.
- If any component of the path falls outside the listed regions, the packet
MUST be discarded.
- Region interpretation is based on prior agreement between the enterprise
and the Cloud Backbone provider (e.g., standard region names, operator-specific
definitions, or standardized Region IDs).
Note:
It is beyond the scope of this document to specify how the Cloud Backbone
enforces this restriction. Mechanisms for identifying region boundaries,
enforcing region-based constraints, and generating alerts or alarm
notifications when traffic violates region restrictions are subject to
implementation decisions and based on prior agreement between the Cloud
Backbone provider and the enterprise.
From: Linda Dunbar
Sent: Monday, August 4, 2025 1:02 PM
To: Joel Halpern <[email protected]>
Subject: RE: [rtgwg] Re: draft-ietf-rtgwg-multisegment-sdwan-04 early Rtgdir
review
Joel,
Thank you!
How about this revision?
(^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^)
( Cloud Backbone )
(+-----+ +----+ +-----+)
+ ---(|CEdge|==| GW1==================== GW2 |)
Direct | (+-----+ +/--\+ +--|--+)
Connect | (^^^^^^^^/^^^^\^^^^^^^^^^^^^^^^^^^^^|^^^)
{-+---} / \ |
{ VPN } / \ +-----+
{-+---} / IPsec Tunnel |CPE10|
+-------+--/--------+ \ +-----+
| / | \ 192.0.2.128/25
++/--+ | +--\-+ 198.51.100.128/25
|CPE1| +--+CPE2|
+----+ +----+
Client Route: 192.0.2.0/26 192.0.2.64/26
198.51.100.0/26 198.51.100.64/26
Figure 2 Multi-Segment SD-WAN Stitching via Cloud Backbone
Linda
From: Joel Halpern <[email protected]<mailto:[email protected]>>
Sent: Sunday, August 3, 2025 7:08 AM
To: Linda Dunbar <[email protected]<mailto:[email protected]>>
Subject: Re: [rtgwg] Re: draft-ietf-rtgwg-multisegment-sdwan-04 early Rtgdir
review
Just for your consideration.
As an example of additional tuning, I think figure 2 could be significantly
clarified. If I understand you properly, the cloud is actually the connection
between GW1 and GW2. So put GW on the left hand side of the cloud, make the
cloud a little larger, and put GW2 on the right hand side of the cloud.
Yours,
Joel
On 8/1/2025 6:42 PM, Linda Dunbar wrote:
Joel,
Thank you very much for the call, making me understand where the confusion is.
Do you think the following revision of the Abstract and the Introduction can
make it clear? Also, we probably should remove the "Include-Sub-TLV" for now
as there are so many issues.
Abstract
This document describes a method for seamlessly interconnecting geographically
separated SD-WAN segments via a Cloud Backbone without requiring Cloud Gateways
(GWs) to decrypt and re-encrypt traffic. By encapsulating IPsec-encrypted
payloads within GENEVE headers (RFC 8926), the approach enables Cloud GWs to
forward encrypted traffic directly between distant Customer Premises Equipment
(CPEs). This reduces processing overhead, improves scalability, and preserves
the confidentiality of enterprise data while ensuring secure and efficient
multi-segment SD-WAN. connectivity.
1. Introduction
Enterprises are increasingly turning to SD-WAN to connect on-premises CPEs with
cloud services, as discussed in detail in [Net2Cloud]. Each SD-WAN segment
typically connects a CPE to its nearest Cloud Gateway (GW). Some of this
traffic terminates at the cloud services and must be decrypted by the Cloud GW.
Other traffic is destined for remote CPEs located in different geographic
regions and only require forwarding across a Cloud Backbone, without decryption.
Multi-segment SD-WAN refers to the architecture in which two or more SD-WAN
segments are interconnected via a Cloud Backbone. This model enables traffic
that originates in one SD-WAN segment to reach a distant CPE through transit
Cloud GWs without decryption. It supports hybrid traffic handling: local
cloud-bound traffic is decrypted by the Cloud GW, while CPE-to-CPE traffic is
forwarded securely across the backbone.
Linda
From: Joel Halpern
<[email protected]><mailto:[email protected]>
Sent: Thursday, July 31, 2025 1:51 PM
To: Linda Dunbar
<[email protected]><mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Cc:
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Subject: Re: draft-ietf-rtgwg-multisegment-sdwan-04 early Rtgdir review
In line, marked <jmh3></jmh3>. Given that I am still confused on these issues,
I suspect even if you end up keeping the behaviors as they are, better
explanation is needed.
Yours,
Joel
On 7/31/2025 4:29 PM, Linda Dunbar wrote:
Joel,
Thank for the reply.
Below are the additional resolutions marked as [Linda3] . Please let me know if
they are acceptable.
Thank you,
Linda
From: Joel Halpern <[email protected]><mailto:[email protected]>
Sent: Wednesday, July 30, 2025 7:38 AM
To: Linda Dunbar
<[email protected]><mailto:[email protected]>; Joel Halpern
<[email protected]><mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Cc:
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Subject: Re: draft-ietf-rtgwg-multisegment-sdwan-04 early Rtgdir review
Trimmed to remove agreements. Context retained for remaining issues. My new
notes marked <jmh2></jmh2>
Yours,
Joel
On 7/29/2025 10:25 PM, Linda Dunbar wrote:
Joel,
Apologies for missing this email through the IETF week.
Thank you very much for the additional comments. Please see below under [Linda
2] for the proposed resolutions. Let us know if they are acceptable.
Linda
From: Joel Halpern <[email protected]><mailto:[email protected]>
Sent: Thursday, July 17, 2025 6:50 PM
To: Linda Dunbar
<[email protected]><mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Cc:
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Subject: Re: draft-ietf-rtgwg-multisegment-sdwan-04 early Rtgdir review
Some of your answers address the raised concerns. However, some of them
indicate I was insufficiently clear, as they do not address the issue. I will
not in line where things are fine,a nd where I think there is still a problem.
I will delimit my comments with <jmh></jmh> in case of indenting problems.
Yours,
Joel
On 7/17/2025 9:12 PM, Linda Dunbar wrote:
Joel,
Thank you very much for reviewing the document and the valuable feedback.
Please see below for the detailed resolutions.
If they are okay with you, we can upload the revision on Monday.
Linda
-----Original Message-----
From: Joel Halpern via Datatracker <[email protected]><mailto:[email protected]>
Sent: Thursday, July 17, 2025 10:46 AM
To: [email protected]<mailto:[email protected]>
Cc:
[email protected]<mailto:[email protected]>;
[email protected]<mailto:[email protected]>
Subject: draft-ietf-rtgwg-multisegment-sdwan-04 early Rtgdir review
Document: draft-ietf-rtgwg-multisegment-sdwan
Title: Multi-segment SD-WAN via Cloud DCs
Reviewer: Joel Halpern
Review result: Not Ready
Hello
I have been selected to do a routing directorate "early" review of this draft.
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-ietf-rtgwg-multisegment-sdwan%2F&data=05%7C02%7Clinda.dunbar%40futurewei.com%7Cde59685e54d84193978008ddc559be0d%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C638883711415554778%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=0rGQhV01eJsDerr6g1jxCzUIkhF5vlwTPOohcWDo3%2Bg%3D&reserved=0<https://datatracker.ietf.org/doc/draft-ietf-rtgwg-multisegment-sdwan/>
The routing directorate will, on request from the working group chair, perform
an "early" review of a draft before it is submitted for publication to the
IESG. The early review can be performed at any time during the draft's lifetime
as a working group document. The purpose of the early review depends on the
stage that the document has reached.
This draft describes the extensions to Geneve to enable using it to
interconnect multiple SD-WAN segments, with particular attention to the case
when the carried payload is IPSec protected.
For more information about the Routing Directorate, please see
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwiki.ietf.org%2Fen%2Fgroup%2Frtg%2FRtgDir&data=05%7C02%7Clinda.dunbar%40futurewei.com%7Cde59685e54d84193978008ddc559be0d%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C0%7C638883711415582636%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=rne%2Fx6T0XCvkiaZPqAx%2F6%2BWGFFOY7%2FXs9tPq%2BmBi5G4%3D&reserved=0<https://wiki.ietf.org/en/group/rtg/RtgDir>
Document: draft-ietf-rtgwg-multisegment-sdwan-04
Reviewer: Joel Halpern
Review Date: 17-July-2025
Intended Status: Proposed Standard
Summary:
I have some minor concerns about this document that I think should be resolved
before it is submitted to the IESG. I also have one major concern that was
flagged by I-D nits <jmh2>resolved</jmh2>. And one major concern where a
procedural element does nto sem to work.
...
Section 4.5 on including specific SD-WAN transitsegments eems an
understandable goal. However, it also seems fraught with failure
potential. In simple topologies, yes, it works. But suppose that the
actual path to the destination is SD-WAN segments A-B-C-D-E. And suppose
the include requirement says "B". When the GENEVE packet arrives at the
C-D boundary, it says that its path must include B. But the path to the
destination from there does not include B. How is the C-D boundary
supposed to know that B has alreay been traversed?
[Linda] Some SD-WAN deployments require specific transit segments to be
included in the end-to-end path for regulatory, security, or service chaining
purposes, rather than for path optimization. The Include-Transit Sub-TLV, an
optional field, is used to signal such requirements. It allows explicitly
specifying a list of Cloud Availability Regions or Zones that a packet must
traverse when forwarded through the Cloud Backbone.
<jmh> I apparently did not explain my concern well enough. I understand why
one would want to mandate that a (or several (specific SD-WAN are traversed.
(I could debate the utility, but operators and customers often want things I
wonder about.) That is not the source of my concern. It is unclear who is
expected to enforce the mandatory traverse case, and how loops are avoided.
Suppose that SD-WAN A is connected to SDS-WAN B, which is connected to SD-WAN C
and D. And D is connect to SD-WAN B and E, where the egress exists. A Geneve
packet is sent with an include indicating that SD-WAN B must be traversed. The
packet happens to go theorugh SD-WAN A to SD-WAN B (meeting the traversal
requirement), then to SD-WAN C, and then to the SD-WAN D. However, by the time
the packet arrives at SD-WAN D, there is no indication in the packet that it
went through SD-WAN B. WHich would seem to require D to send the packet back
to SD-WAN B. Which is clearly not what is desired. How does the gateway to
SD-WAN D know that it si fine to keep forwarding towards the destination
(egress from E) rather than returning the packet to B? I could understand how
it worked if there was also a route record. Or if the Include was removed once
it was satisfied. But the draft does not call for either of those behaviors.
</jmh>
[Linda 2] Your illustration regarding potential loops and the lack of explicit
enforcement mechanisms is entirely valid. However, as noted in the draft, it is
out of scope for the IETF (and this document) to define how the cloud backbone
enforces client-specified traversal preferences. The Include-Transit Sub-TLV is
intended only to convey the client's desired intent or preferences. How about
adding this sentence to the section:
"..., this indication reflects preference only and does not imply any guarantee
of traversal. Enforcement of the include list is outside the scope of this
document and must be realized, if needed, through mutual agreement and
provider-specific mechanisms.."
<jmh2>Note that in the end I am not the one you have to satisfy. But, from
where I sit, a behavioral marking that can't be acted upon without additional
protocol behaviors (which ar enot obviously valid to me) is not a marking we
can have in an RFC. It does not lead to interopeable implementation. </jmh2>
[Linda3] Your points are well taken. That said, there is precedent for this
approach in RFCs 8670, 7752, and 8306, where protocol elements convey intent or
preference, but their interpretation and enforcement are left to local policy
or controller behavior. In the same spirit, the Include-Transit Sub-TLV
expresses the originator CPE's intent to prefer certain transit regions, while
how the Cloud Backbone enforces that preference is intentionally left out of
scope for this document..
<jmh3>The problem I have is that I can't figure out how, if the gateways are
forwarding packets simply as IP encapsulated Geneve packets, they can conform
to the intent. There is an assumption about how the gateways do their job, and
how they are controlled, that is unstated. This seems to leave a high
probability of inconssitent implementation. As far as I can tell, to comply
with the policy, the gateway into the included domain would have to modify the
geneve packet to remvoe the intent. Which, as I understand it, is against the
RFCs.
</jmh3>
Minor Issues:
...
In section 5, in describing the Ingress GW processing, the text is written
as if the outer IP destination address will always become the egress
gateway. As I understand it, if the path goes through multiple SD-WAN, the
outer IP address at each stage is that of the next gateway? Could the text
be rewritten to make that clear. Also, doesn't this imply there is a
"transit gateway" case as well as ingress and egress?
[Linda] The GENEVE header remains during transit across the Cloud Backbone and
is removed by the egress Cloud Gateway before the packet is forwarded to the
destination CPE. The packet is forwarded natively on the final SD-WAN segment
(egress GW to destination CPE) without GENEVE encapsulation.
<jmh>I am still missing something. It may be that I am misunderstanding the
interaction between mutli-segment SD-WAN and GENEVE. Suppose that we have GW1 -
SDWAN A - GW 2 - SDWAN B - GW-3. When the packet arrives at GW-1 with the
multi-segment SD-WAN option, GW1 decides (subject to the constraints of the
options in the packet) that the packet should go to GW2 in orderr to get to
GW3. As I understand it, GW1 will replace the outer IP destination (which was
GW-1 upon arrival) with the IP address of GW2. But the text says that it will
replace the destination address with the egress IP address (GW-3, or maybe
something beyond that.) </jmh>
<jmh2>I do not see a response to this issue</jmh2>
[Linda3] Sorry for missing this one. The ingress GW determines the appropriate
egress Cloud GW based on its internal logic or policy, or via the Egress GW
Sub-TLV included in the Multi-Segment Metadata.
The destination address in the outer IP header of the GENEVE packet should be
determined by the Cloud Backbone. This address is intended to reach the egress
Cloud GW identified by the Egress GW Sub-TLV (if present), or the optimal
egress GW selected based on the destination CPE address.
Changed the text to the following:
<jmh3>We are in amulti-segment SD-WAN. GWI is the ingress, GWE is the Egress.
and the selected path is through GW2 and GW3. If GWI changes the outer IP
address to be the address of GWE, then how does it direct the packet to GW2,
and thence to GW3? This may be related to the above issue of an assumed
unspecified behavior at SD-WAN gateways. </jmh3>
I do not know if this is a major concern, a minor concern, or merely a
confused reviewer. There is a description in section 9 of an attack to
steal data service (conceptually, an understandable problem.) However, I
am unable to figure out what set of access to what set of places the
attacker must have, nor how adding authentication to the CPE / GW exchange
would actually help prevent this attack. In part this is because the
attack appears underspecified, and in part this is because the remediation
appears underspecified.
[Linda] does it help to add this sentence to the introduction?
In this document, "multi-segment SD-WAN" refers specifically to deployments
with two SD-WAN edge segments: one from the originating CPE to the ingress
Cloud Gateway, and one from the egress Cloud Gateway to the destination CPE.
There may be additional SD-WAN segments or forwarding domains between the
ingress and egress Cloud Gateways across the Cloud Backbone, but their internal
behavior is out of scope for this specification.
<jmh>Nope, sorry. That does not tell me what the security threat is that leads
to wanting authentication. It also does not tell me how the authentication is
actually to be done. (I naive reading is that you are inventing an
authentication extension, with insufficient specificity, to some unspecified
protocol between the CPE and the first SD-WAN gateway.)
</jmh>
[Linda2] Section 9 describes the consequence (data theft) but does not clearly
articulate the security threat or the authentication model. To clarify:
1. The threat is that a malicious or misconfigured CPE could inject SD-WAN
metadata intended for another tenant, attempting to use or redirect traffic
through paths it is not authorized for.
2. Authentication is needed to verify the origin and legitimacy of the
SD-WAN metadata, ensuring it is associated with an authorized endpoint.
<jmh2>At the very least, this needs to be better explained in the draft. I
think the issue is larger. Adding security to the CPE-SDWAN connection seems
to be largely irrelevant if that CPE is compromised. Additionally, if the
SDWAN gateway accepts traffic from arbitrary devices, taht would seem to be an
unerlying SDWAN problem, not a multi-segment security issue.</jmh2>
[Linda3] do you think the following text for Bullet c) under Section 9.1 can
reflect those two key points?
(c) Potential stealing of Cloud Backbone bandwidth:
A threat actor, such as a malicious or misconfigured C-PE, could inject SD-WAN
metadata intended for another tenant, attempting to redirect traffic through
Cloud Backbone paths it is not authorized to use. For example, a legitimate
Cloud subscriber pays for Cloud Backbone transport between CPE-A and CPE-B. An
attacker, who controls two locations (Node-A and Node-B), might construct a
packet with CPE-A's address as the source and include CPE-B in the SD-WAN
Endpoint Sub-TLV. The packet, when sent to the ingress Cloud GW, would appear
to be a legitimate flow between CPE-A and CPE-B. After exiting the egress Cloud
GW, the attacker could rewrite the outer addresses to resume communication
between Node-A and Node-B, effectively tunneling traffic via the Cloud Backbone
without authorization or payment.
To prevent such misuse, it is necessary to authenticate and verify the origin
and legitimacy of the SD-WAN metadata to ensure it is associated with an
authorized and registered CPE endpoint. This validation protects against both
spoofing and cross-tenant policy violations. While it is not necessary for
Cloud GWs to decrypt or re-encrypt payloads, they must enforce metadata
integrity using authentication mechanisms such as those defined in [RFC2403]
and [RFC2404]. These mechanisms rely on cryptographic hashing algorithms (e.g.,
SHA2 224/256/384/512) as part of a Hashed Message Authentication Code (HMAC) to
validate packet authenticity.
<jmh3>This sseems to be an underlying SD-WAN security problem, not a
multi-segment problem. It seems like the same attack could be done in a
single-segment SD-WAN? If that is true, and if the existing SD-WAN RFCs have
this problem, then I would think noting in the remediation that this applies
more broadly would seem to help. If I am wrong about the applicability, some
indication of why it is only a problem in multi-segment would be desirable.
</jmh3>
<jmh>Thanks.</jmh>
_______________________________________________
rtgwg mailing list -- [email protected]<mailto:[email protected]>
To unsubscribe send an email to
[email protected]<mailto:[email protected]>
_______________________________________________
rtgwg mailing list -- [email protected]
To unsubscribe send an email to [email protected]
