"after an external request for that origin." With serve_stale_for=0 and the DNS record is expired,
- probe() will return nullptr - getby() does not handle nullptr response from probe, and will just schedule probeEvent after 20ms. (This seems less than ideal. Why not immediate?) - probeEvent() calls do_dns() and sets the cont to wait for DNS response. I know Verizon and Apple use serve_stale_for > 0, so this situation might not be optimized or tested often. When it uses the OOR origin can you tell the age of the DNS record it used, or prove that the DNS, in fact, told ATS to use a different host? On Wed, Mar 13, 2019 at 10:37 AM Nishant Gaurav <nishant16...@gmail.com> wrote: > Hi Aaron, > > Thank you so much your reply. I had verified that I am not accidentally > setting serve_stale_for. > > $ traffic_ctl config get proxy.config.hostdb.serve_stale_for > proxy.config.hostdb.serve_stale_for: 0 > > $ traffic_ctl config get proxy.config.hostdb.verify_after > proxy.config.hostdb.verify_after: 1 > > I also verified that even if the field `stale` is `yes` (along with > `timedout` = `yes`), hostdb still doesn't refresh the records on its own > but waits for a request for that origin to come in. Just to confirm, when > you say "It should be forced to lookup the DNS and use the new record > unless you are using proxy.config.hostdb.serve_stale_for, that is the only > reason i can think you are still going to the OOR host." - You meant forced > to look up on its own (without any external request for that origin) or > after an external request for that origin. > > Regards, > Nishant > > From: Aaron Canary <acan...@verizonmedia.com.invalid> > Date: 2019年3月12日周二 下午1:52 > Subject: Re: Question regarding hostdb > To: <dev@trafficserver.apache.org> > Cc: Jon Patterson <jopatter...@linkedin.com> > > > verify_after does not poll or refresh records unless prompted by a request > of that hostname. > > It should be forced to lookup the DNS and use the new record unless you are > using proxy.config.hostdb.serve_stale_for, that is the only reason i can > think you are still going to the OOR host. > > I'd guess the design decision there was that if traffic to that host is > THAT low, then adding the latency of a DNS lookup doesn't matter. > > > On Tue, Mar 12, 2019 at 2:49 PM Nishant Gaurav <ngau...@linkedin.com> > wrote: > > > Hi, > > > > I was trying to debug an issue where a low QPS service's host keeps > > getting traffic even after it goes out of rotation. While debugging I > > observed that hostdb record for any given origin doesn't get updated > unless > > trafficserver gets a call corresponding to that origin. This happen even > > when `ttl` field of that origin's hostdb record is negative (`timedout` > > field having value `yes`). > > > > So, let's say that 1 host of an origin went out of rotation (OOR) at time > > T seconds, and the first call that the trafficserver got for that origin > > (after the host going OOR) was at time T + 100 seconds. Since the hostdb > > records have not been updated up until T + 100, there is a chance that > the > > first call can go to the OOR host. As soon as the first call comes hostdb > > record for that origin gets updated (by making a call to DNS) and > > everything works fine next call onwards. As I said earlier this only > > becomes a problem with services which are having very less QPS. > > > > I just wanted to ask if there is a way to overcome this problem so that > > the hostdb record can be updated every time ttl expires? I tried to work > > with proxy.config.hostdb.verify_after (by setting it to a low value) - > but > > that didn't solve it for me. > > > > Regards, > > Nishant > > >
