On 3 Apr 2020, at 15:08, Kurt Jaeger <p...@freebsd.org> wrote: > > Hi! > >> It is true for both IPv4 and IPv6, and for Russia and Germany (I can >> not test from USA, though). > > I tried it from freefall, same problem.
Typically one of those "Endpoint Protection" products, they have some sort of whitelist or blacklist of bad user agents. Using "curl/7.68.0" as user agent works: $ fetch -vvv --user-agent="curl/7.68.0" http://www.ti.com/lit/ug/slau646e/slau646e.pdf scheme: "http" user: "" password: "" host: "www.ti.com" port: "0" document: "/lit/ug/slau646e/slau646e.pdf" ---> www.ti.com:80 resolving server address: www.ti.com:80 requesting http://www.ti.com/lit/ug/slau646e/slau646e.pdf >>> GET /lit/ug/slau646e/slau646e.pdf HTTP/1.1 >>> Host: www.ti.com >>> Accept: */* >>> User-Agent: curl/7.68.0 >>> Connection: close >>> <<< HTTP/1.1 200 OK <<< Server: Apache <<< Last-Modified: Wed, 26 Jun 2019 21:34:37 GMT <<< ETag: "1551ab-58c40cfeb6c6f" last modified: [2019-06-26 21:34:37] <<< Accept-Ranges: bytes <<< Content-Length: 1397163 <<< Cache-Control: max-age=60 content length: [1397163] <<< Expires: Thu, 30 Jan 2020 09:27:51 GMT <<< Content-Type: application/pdf <<< Date: Fri, 03 Apr 2020 17:30:20 GMT <<< Connection: close <<< Set-Cookie: ti_geo=country=XX|city=YYYYYYY|continent=EU|tc_ip=n.m.o.p; path=/; domain=.ti.com <<< Set-Cookie: ti_rid=3110b500; path=/; domain=.ti.com <<< Set-Cookie: ti_ua=curl%2f7.68.0; path=/; domain=.ti.com <<< Set-Cookie: ti_ak_id=67fba42067fba42067fba4205e8772ac3110b500; expires=Sat, 03-Apr-2021 17:30:20 GMT; path=/; domain=.ti.com <<< Set-Cookie: ti_ridh=expired; path=/; domain=ti.com; expires=Thu, 1-Aug-2019 06:14:04 GMT <<< Set-Cookie: ti_bm=; path=/; domain=.ti.com <<< offset 0, length -1, size -1, clength 1397163 local size / mtime: 1397163 / 1585934997 remote size / mtime: 1397163 / 1561584877 slau646e.pdf 1364 kB 15 MBps 00s You can immediately see they're trying to violate cookie laws. ;-) The "ti_ua" cookie stores the user agent, and I guess "tk_ak_id" is the Akamai ID (a.k.a. personally identifiable information). -Dimitry
signature.asc
Description: Message signed with OpenPGP
