Hi Diego, You're absolutely right to point out the flaw in my snippet, thanks for catching that. That said, my goal wasn't to provide a complete solution, but rather a minimal example to highlight the core issue. Judging from your ability to spot the edge case, I’m sure you’re more than capable of refining the logic, if you want feel free to share it so we can refine it together.
Regarding the point about query parameters. I think we’re actually in agreement. Just to clarify where I was coming from: Query parameters (the part after ?) are generally treated as opaque strings by ParseRequestURI itself with respect to their *content's* encoding validity at that stage, though u.Query() would later parse RawQuery. This is why %2F or even Ñ%2FÑ in a query value is passed through to RawQuery without ParseRequestURI altering its fundamental encoding or structure. That’s really what I was trying to isolate in that check the silent, lossy transformation that happens with EscapedPath() when certain characters are present in the path. I agree though that any detection logic needs to respect the boundary between the path and the query to avoid false positives. If you or anyone else ends up with a clean workaround or pattern, I’d love to hear it. Kind regards, Alexander ___ https://go.dev/play/p/jPoTJfuqrJq ```go package main import ( "fmt" "net/url" ) func show(raw string) { fmt.Println("Raw URL: ", raw) u, err := url.ParseRequestURI(raw) if err != nil { fmt.Println(" → parse error:", err) fmt.Println() return } fmt.Println(" Path: ", u.Path) fmt.Println(" RawPath: ", u.RawPath) fmt.Println(" EscapedPath: ", u.EscapedPath()) fmt.Println(" Query: ", u.RawQuery) fmt.Println() } func main() { examples := []string{ // 1) percent-encoded slash in path "http://example.com/foo%2Fbar";, // 2) percent-encoded slash in query "http://example.com/?thing=%2Ffoo";, // 3) non-ASCII + percent-encoded slash in path "http://example.com/Ñ%2FÑ <http://example.com/%C3%91%2F%C3%91>", // 4) malformed encoding in path "http://example.com/bad%ZZpath";, // 5) malformed encoding in query "http://example.com/?q=bad%ZZ";, // 6) malformed encoding using Ñ%2FÑ in query "http://example.com/?q=Ñ%2FÑ <http://example.com/?q=%C3%91%2F%C3%91> ", } for _, raw := range examples { show(raw) } } ``` ____ Raw URL: http://example.com/foo%2Fbar Path: /foo/bar RawPath: /foo%2Fbar EscapedPath: /foo%2Fbar Query: Raw URL: http://example.com/?thing=%2Ffoo Path: / RawPath: EscapedPath: / Query: thing=%2Ffoo Raw URL: http://example.com/Ñ%2FÑ <http://example.com/%C3%91%2F%C3%91> Path: /Ñ/Ñ RawPath: /Ñ%2FÑ EscapedPath: /%C3%91/%C3%91 Query: Raw URL: http://example.com/bad%ZZpath → parse error: parse "http://example.com/bad%ZZpath": invalid URL escape "%ZZ" Raw URL: http://example.com/?q=bad%ZZ Path: / RawPath: EscapedPath: / Query: q=bad%ZZ Raw URL: http://example.com/?q=Ñ%2FÑ <http://example.com/?q=%C3%91%2F%C3%91> Path: / RawPath: EscapedPath: / Query: q=Ñ%2FÑ Am Do., 15. Mai 2025 um 17:53 Uhr schrieb 'Diego Molina' via golang-nuts < golang-nuts@googlegroups.com>: > Hi Alexander, thank you for your response. > > In your example, strings.Contains(strings.ToLower(original), "%2f") will > give a false positive if we have passed a "/" in a query param since it > would also be url-encoded, and this is valid and expected. You will come > across this in your new API if you need to pass a "/" in a query param > value, or even if it's not part of your own API since it's common to find > back URLs in query params. Example URL: "/?thing=%2Fasddsa". > > Kind regards, > > > > On Sunday, May 4, 2025 at 5:26:43 PM UTC-3 Alexander Ertli wrote: > >> EscapedPath() sometimes re-generates the escaped path, especially if the >> original path contains unencoded characters like Ñ or literal spaces. >> When it does this reconstruction, it works from the decoded path segments, >> which is likely why the original %2F encoding was lost in your example. >> This seems to be intentional—and probably a valid interpretation of RFC >> 3986, even if one with side effects. >> >> I also came across a related issue, and the solution ended up being a >> major API change—switching to query parameters in our case. The cloud >> provider was escaping the path exactly as you described before it hit our >> backend. (So maybe reconsider whether you really want to use >> EscapedPath() if you're building a proxy or router.) >> >> That said, I'm sorry I only have a workaround to offer. Here's one: >> ```go >> func check(original string) { >> u, err := url.ParseRequestURI(original) >> noerr(err) >> >> escapedPath := u.EscapedPath() >> hasEncodedSlash := strings.Contains(strings.ToLower(original), "%2f") >> isCrafted := hasEncodedSlash && (original != escapedPath) >> >> fmt.Println("Original path: ", original) >> fmt.Println("Escaped path: ", escapedPath) >> fmt.Println("Anomaly Detected:", isCrafted) >> fmt.Println() >> } >> ``` >> If someone knows a better fix, I'd really appreciate hearing it. >> >> Am Mi., 30. Apr. 2025 um 00:08 Uhr schrieb Diego Augusto Molina < >> diegoaugu...@gmail.com>: >> >>> Errata: In the second HTTP example I gave: >>> >>> HTTP /x%2Fx HTTP/1.1 >>> Host example.com >>> >>> EscapedPath() will return "/x%2Fx", not "/%2F". >>> >>> Kind regards, >>> >>> On Tuesday, 29 April 2025 at 18:58:59 UTC-3 Diego Augusto Molina wrote: >>> >>>> Hi everyone, thank you in advance for reading. >>>> >>>> I was trying to use EscapedPath() to do path-based routing in an HTTP >>>> server, where some requests need to be forwarded to another service. I >>>> found that an invalid character in some parts of the path of the Request >>>> URI passed in the HTTP Request Line would not cause an error, but instead >>>> would be silently accepted and when I had to forward the request then I >>>> would be forwarding something I wasn't expecting to. >>>> >>>> When the path contains all valid path characters then using >>>> EscapedPath() and String() methods work as (I would have) expected, and I >>>> can do things like path.Clean to remove "..", ".", and "/////" from the >>>> path. But if I put something like "Ñ" in the path, then EscapedPath() (and >>>> String(), as it calls EscapedPath()) will interpret other %-encoded >>>> sequences in the passed path. So if I send an HTTP request like: >>>> >>>> HTTP /x%2Fx HTTP/1.1 >>>> Host example.com >>>> >>>> Then EscapedPath() will return "/%2F", which is to say a path with a >>>> single path element: "x/x". But if I send a request like: >>>> >>>> HTTP /Ñ%2FÑ HTTP/1.1 >>>> Host example.com >>>> >>>> Then EscapedPath() will return "/%C3%91/%C3%91", which is to say a >>>> path with two path elements: "Ñ" and "Ñ", whereas I would expect a >>>> parsing error and no handler being called. Note also that my "%2F" was >>>> interpreted. When forwarding the request to another service, this could be >>>> a problem because it's interpreting user input and the user could >>>> potentially traverse the proxied service. >>>> >>>> Here is a small Playground to illustrate: >>>> https://go.dev/play/p/ySJwVtvHHQF >>>> >>>> I thought of a few workarounds on the language-user side: >>>> >>>> 1. Manually writing a validator to know if the path is valid and >>>> return an error. Prone to error. >>>> 2. "go:linkname ...validEncoded" instead of writing my own. >>>> Definitely not on my plans. >>>> 3. Set RawPath to the empty string, and live with the fact that the >>>> user can send a %2F and that it will be interpreted as a literal "/" >>>> separating path elements, but as long as every part of the program and >>>> all >>>> systems that do something different based on the path see the same >>>> thing, >>>> then that's probably better. >>>> >>>> >>>> I read the code in Chi and Gin router libraries to see if others had >>>> the same problem, and both of them appear to have related issues reported, >>>> but so far it appears to be a contentious or otherwise unresolved subject. >>>> The following are probably related: >>>> >>>> - https://github.com/go-chi/chi/issues/641 >>>> - https://github.com/go-chi/chi/issues/642 >>>> - https://github.com/go-chi/chi/issues/832 >>>> - https://github.com/gin-gonic/gin/issues/4033 >>>> >>>> Here is also a related reddit thread: >>>> https://www.reddit.com/r/golang/comments/1gbcgpx/confused_about_urlpath_urlrawpath_and/?rdt=64426 >>>> >>>> Note that I took the assumption on what is "valid" based on my >>>> interpretation of RFC 3986 "Uniform Resource Identifier (URI): Generic >>>> Syntax" §3.3 (https://www.rfc-editor.org/rfc/rfc3986#section-3.3), >>>> which is also mentioned in the source code of the package. >>>> >>>> Please, let me know your thoughts, if I'm missing anything, or if there >>>> are better alternatives. >>>> >>>> Thank you. >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "golang-nuts" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to golang-nuts...@googlegroups.com. >>> To view this discussion visit >>> https://groups.google.com/d/msgid/golang-nuts/c4ffe669-cfbd-4774-a327-6ec8c974fdccn%40googlegroups.com >>> <https://groups.google.com/d/msgid/golang-nuts/c4ffe669-cfbd-4774-a327-6ec8c974fdccn%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com. > To view this discussion visit > https://groups.google.com/d/msgid/golang-nuts/53338881-ea41-465b-a63d-01bf1fadc5ecn%40googlegroups.com > <https://groups.google.com/d/msgid/golang-nuts/53338881-ea41-465b-a63d-01bf1fadc5ecn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/golang-nuts/CAC4E5ZnGiNGO7Ag0UtHOBTZirLN2_7mODdO_YCyyJhmSsbuzgA%40mail.gmail.com.
