On Friday December 23 2016 09:35:23 Ryan Schmidt wrote: >I'm sure the server administrator did it intentionally, but I consider it an >error for the server administrator to have done so.
The administrator probably disagrees. Or the organisation s/he depends on. Is there any text that states sites have to allow trawling of their sites, or downloading of their full content by bots? If not there can be no question of error.
