In testing the current code behavior, I've discovered that TSHttpHdrEffectiveUrlBufGet() omits the scheme when called on the request to server:
Transaction: event=TS_EVENT_HTTP_SEND_REQUEST_HDR TSHttpTxnEffectiveUrlStringGet(): http://127.0.0.1:61000/ TSHttpHdrEffectiveUrlBufGet(): 127.0.0.1:61000/ TSUrlSchemeGet(): failed to get URL scheme That's not what we want is it? On Wed, Oct 7, 2020 at 10:10 AM Alan Carroll <solidwallofc...@verizonmedia.com.invalid> wrote: > The problem arises on the proxy request, not the inbound request. > > TSUrlRawSchemeGet should return an empty string for the first case. > > On Tue, Oct 6, 2020 at 3:22 PM Walt Karas <wka...@verizonmedia.com > .invalid> > wrote: > > > So, when I call the current TSUrlSchemeGet() for these two requests: > > > > printf "GET / HTTP/1.1\r\nHost: mYhOsT.teSt:61000\r\n\r\n" | nc localhost > > 61001 > > printf "GET > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__mYhOsT.teSt-3A61000_&d=DwIFaQ&c=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY&r=5nE_8e-Jc1t5vF6GVeub9BCN4FzSc_6kU7_mjSiUrDs&m=tHbcxwP-7R4CDqWxQtN0lU5SU0iqTJtPosNC-K_tQ6Q&s=oi7IufY5vsl5r1JR9oGUyIHWUsG5ANRXj-SZXx67r2g&e= > > HTTP/1.1\r\n\r\n" | nc localhost 61001 > > > > I get "http" for both. Does that mean it's already working as desired? > > Should TSUrlRawSchemeGet() return an empty string for the first request? > > > > On Wed, Sep 30, 2020 at 11:14 AM Alan Carroll > > <solidwallofc...@verizonmedia.com.invalid> wrote: > > > > > There has been a lot of discussion on this and the related change for > the > > > URL port. You can see some of this on the ASF slack channel, but I will > > > summarize in this note. > > > > > > Leif objected to this change and the current compromise is to > > > > > > 1. Change the current TSUrlSchemeGet to do what was proposed for > > > TSHttpHdrSchemeGet, that is if the scheme is not literally in the URL, > > the > > > value in the internal member is used to return the WKS for the value. > > > 2. Add TSUrlRawSchemeGet which does what TSUrlSchemeGet does now. > > > > > > In practice, this would be renaming TSUrlSchemeGet to > TSUrlRawSchemeGet, > > > and add TSUrlSchemeGet to do the "clever" thing. > > > > > > In the same way, do this for TSUrlPortGet and TSUrlRawPortGet. > > > > > > In essence TSUrlSchemeGet and TSUrlPortGet return the effective value, > > and > > > the "Raw" variants get the literal value. > > > > > > This may need to be updated in the future for HTTP/2 outbound where the > > > scheme can be set in an HTTP/2 field. OTOH if it is a field that could > be > > > handled the same way as for "Host" currently, which is by leaving the. > > > burden of checking that to the plugin. > > > > > > On Tue, Sep 29, 2020 at 9:38 AM Alan Carroll < > > > solidwallofc...@verizonmedia.com> wrote: > > > > > > > I don't see how this would depend on a cache hit or miss. If two > > requests > > > > map to the same object, that's the cache key, not the request scheme. > > > This > > > > returns the scheme in hdr->m_http->req.m_url_impl.scheme or > > > > hdr->m_http->req.m_url_impl.m_url_type if the former is nullptr. > > > > > > > > The point here is to provide access to data that is in the core but > not > > > > currently available to a plugin, that is > > > > hdr->m_http->req.m_url_impl.m_url_type. Consider the case where a > user > > > > agent sends a request for " > > > > > > https://urldefense.com/v3/__http://delain.nl/lucidity.html__;!!Op6eflyXZCqGR5I!SNVEMa63RJNusBdmJy0FRUobklIbRbCFbc9t2EuMlXaoYK8z_k0IRNAhMAOGT8S8HA$ > > > ". When the > > > > proxy request is created, it will have only "lucidity.html" in the > > > request > > > > URL. Yet, unless the scheme was explicitly changed via a plugin or > > remap, > > > > the core still knows it's an HTTP request. But how could a plugin > know? > > > > TSUrlSchemeGet will return a nullptr. In this case, however, > > > > TSHttpHdrSchemeGet would return "http". > > > > > > > > This is very similar to TSHttpHdrHostGet, and is useful for the same > > > > reasons. > > > > > > > > > > > > On Mon, Sep 28, 2020 at 9:13 PM Leif Hedstrom <zw...@apache.org> > > wrote: > > > > > > > >> Also what’s the semantic here when both http:// and https:// URLs > > map > > > >> to the same cached object ? The first cached request specifies the > > > scheme? > > > >> This seems confusing at best... or are we talking about the scheme > as > > it > > > >> goes to origin (which would have to be the same for both). > > > >> > > > >> Seems like a remap plugin could just look at the FromURL (or ToURL) > > > which > > > >> should have the scheme, rather than the cached data. And no new APIs > > > >> needed. For a global plugins it’s less obvious, but same issues o > > think? > > > >> > > > >> — Leif > > > >> > > > >> > On Sep 28, 2020, at 20:05, Leif Hedstrom <zw...@apache.org> > wrote: > > > >> > > > > >> > The point here being to make a new API that replaces the old, > > without > > > >> breaking compatibility? And this new API has special semantics on a > > > cache > > > >> hit vs cache miss? > > > >> > > > > >> > This seems pretty convoluted, making it difficult for plugin > writers > > > to > > > >> use the right API... > > > >> > > > > >> > — Leif > > > >> > > > > >> >> On Sep 28, 2020, at 19:49, Brian Neradt <brian.ner...@gmail.com> > > > >> wrote: > > > >> >> > > > >> >> +1 > > > >> >> > > > >> >> Traffic Dump can make use of this. > > > >> >> > > > >> >>> On Mon, Sep 28, 2020 at 7:38 PM Walt Karas < > > wka...@verizonmedia.com > > > >> .invalid> > > > >> >>> wrote: > > > >> >>> > > > >> >>> This should get the scheme for the request. This differs from > > > >> >>> `TSUrlSchemeGet` in that it gets the scheme even if it is not in > > the > > > >> URL of > > > >> >>> the request. For most proxy requests, the ATS core will remove > the > > > >> host and > > > >> >>> scheme in the request while tracking it internally. In such a > > case a > > > >> plugin > > > >> >>> cannot discover that information, a problem this API would fix. > > > >> >>> > > > >> >>> If the scheme is in the request URL, return that. Otherwise > > return a > > > >> scheme > > > >> >>> that corresponds to the internally stored scheme. > > > >> >>> > > > >> >> > > > >> >> > > > >> >> -- > > > >> >> "Come to Me, all who are weary and heavy-laden, and I will > > > >> >> give you rest. Take My yoke upon you and learn from Me, for > > > >> >> I am gentle and humble in heart, and you will find rest for > > > >> >> your souls. For My yoke is easy and My burden is light." > > > >> >> > > > >> >> ~ Matthew 11:28-30 > > > >> > > > >> > > > > > >
