7;m implementing a simple web crawler with curl and want to retrieve the
>> > Last-Modified header so I can implement a sensible recrawl policy. I've
>> found
>> > https://curl.haxx.se/libcurl/c/getinfo.html which is a nice easy way to
>> > retrieve the Content-Ty
On Thu, May 21, 2020 at 4:18 PM Dan Fandrich via curl-library <
curl-library@cool.haxx.se> wrote:
> On Thu, May 21, 2020 at 03:46:33PM +0100, James Read via curl-library
> wrote:
> > I'm implementing a simple web crawler with curl and want to retrieve the
> >
On Thu, May 21, 2020 at 03:46:33PM +0100, James Read via curl-library wrote:
> I'm implementing a simple web crawler with curl and want to retrieve the
> Last-Modified header so I can implement a sensible recrawl policy. I've found
> https://curl.haxx.se/libcurl/c/getinfo.h
Hi,
I'm implementing a simple web crawler with curl and want to retrieve the
Last-Modified header so I can implement a sensible recrawl policy. I've
found https://curl.haxx.se/libcurl/c/getinfo.html which is a nice easy way
to retrieve the Content-Type header. Is there a similarly e
