Re: Implementing Rate-limiting in forward proxy mode

Uhm! why async timers? You'd want to implement a leaky/token bucket per site. Check out... https://github.com/rigtorp/TokenBucket It's single header file lock free implementation for token bucket and it works very well... On Fri, Jun 21, 2019 at 7:38 PM Weixi Li (BLOOMBERG/ PRINCETON) < wli...@

Re: Implementing Rate-limiting in forward proxy mode

What a great community! So many good tips in such a short time! Especially the atscppai, I would've never noticed it. the async examples look very promising. It looks like the following might be necessary (let me know if I'm wrong): * A hook to TS_HTTP_SEND_REQUEST_HDR_HOOK * A map of queues (o

Re: Implementing Rate-limiting in forward proxy mode

I have implemented rate-limit in my plugin using atscppapi. We are using ats in security context for mitigation. If the request matches certain criteria (ip, method, host, uri and header values) then we apply rate-limit to that ip. Dk. > On Jun 21, 2019, at 3:15 PM, Leif Hedstrom wrote: > >

Re: Implementing Rate-limiting in forward proxy mode

Additionally, fyi, there’s an indirect rate limiting support inbuilt into the ATS core via limiting concurrent connections per origin. Refer ‘proxy.config.http.per_server.connection.max’ and ‘proxy.config.http.per_server.connection.queue_size’. You may want to check that out, and if it doesn’t

Re: Implementing Rate-limiting in forward proxy mode

There is no need to fork the code to create a plugin: https://docs.trafficserver.apache.org/en/8.0.x/developer-guide/plugins/getting-started/a-simple-plugin.en.html Once it's working and tested, it would be appreciated if you would offer it for addition in plugins/experimental under TS's Apache li

Re: Implementing Rate-limiting in forward proxy mode

> On Jun 21, 2019, at 16:09, Weixi Li (BLOOMBERG/ PRINCETON) > wrote: > > Hi team, > > We are experimenting with ATS in *forward* proxy mode. Our use-case requires > a rate-limiting component that enforces rules based on the destination. > > For example: > > For all incoming requests tar

Implementing Rate-limiting in forward proxy mode

Hi team, We are experimenting with ATS in *forward* proxy mode. Our use-case requires a rate-limiting component that enforces rules based on the destination. For example: For all incoming requests targeting "www.cnn.com", we want to limit the outgoing rate to be 10 requests per minute; for "w

Re: Fetching URL contents in a plugin

> > Yes, it's a problem that TSIOBufferReaderRead doesn't exist yet. What you > should do is > TSIOBufferBlock block = TSIOBufferStart(iobuff); char const* text = TSIOBufferBlockReadStart(block); text now points at the URL string. For the length you can use TSUrlLengthGet() or TSIOBufferReadAvail

Re: Fetching URL contents in a plugin

The TSUrl* functions are grabbing the URL itself from an MBuf/IOBuf, but my goal is to download the contents of that URL into a char*. I think I’m pretty close at this point as I have the downloaded response body in a IOBuffer, but can't figure how to copy that IOBuf into a char*. TSIOBufferR

Re: Fetching URL contents in a plugin

There's TSUrlStringGet(), which unfortunately returns an allocated buffer. Internally there is url_string_get_buff but it's apparently not accessible from the C API. I should fix that. On Fri, Jun 21, 2019 at 11:21 AM Alan Carroll < solidwallofc...@verizonmedia.com> wrote: > Does TSUrlPrint not w

Re: Fetching URL contents in a plugin

Does TSUrlPrint not work for you? If you set up the TSIOBuffer to have a large block size (say >= 128K) it should fit in a single block. We also might want to push on adding TSIOBufferReaderRead() which would make that easier as well. On Fri, Jun 21, 2019 at 11:11 AM Eric Friedrich -X (efriedri -

Re: Fetching URL contents in a plugin

Yeah thats where I was looking to find the example. I’m basically cribbing the state machine from there with a few modifications. —Eric > On Jun 21, 2019, at 11:29 AM, Leif Hedstrom wrote: > > > >> On Jun 21, 2019, at 06:25, Eric Friedrich -X (efriedri - TRITON UK BIDCO >> LIMITED c/o Alte

Re: Fetching URL contents in a plugin

> On Jun 21, 2019, at 06:25, Eric Friedrich -X (efriedri - TRITON UK BIDCO > LIMITED c/o Alter Domus (UK) Limited -OBO at Cisco) > wrote: > > Thanks- > On receiving a request for an object (say at URL1), I’m trying to fetch a > different file (URL2) that contains some authorization data to

Re: Fetching URL contents in a plugin

It's probably easier to use the CPP API: https://github.com/apache/trafficserver/blob/master/include/tscpp/api/InterceptPlugin.h But there are rumors of lower performance with the CPP API. On Fri, Jun 21, 2019 at 10:06 AM Walt Karas wrote: > Sounds like maybe you want an intercept plugin: > htt

Re: Fetching URL contents in a plugin

Sounds like maybe you want an intercept plugin: https://docs.trafficserver.apache.org/en/8.0.x/developer-guide/plugins/hooks-and-transactions/intercepting-http-transactions.en.html

Re: Fetching URL contents in a plugin

Thanks- On receiving a request for an object (say at URL1), I’m trying to fetch a different file (URL2) that contains some authorization data to determine if the original request should be allowed. My hope was to find a more compact way to build a new request, do a TSHttpConnect() and then r