On 7/25/14 6:31 PM, Michael Brunnbauer wrote:
Hello Kingsley,

On Fri, Jul 25, 2014 at 05:47:58PM -0400, Kingsley Idehen wrote:
>When you have a sense of the identity of an Agent and on behalf of whom it
>is operating, you can use RDF based Linked Data to construct and enforce
>usage policies.
<sarcasm>
Yes. Every "Agent" that does not use WebID-TLS supporting every possible
RDF serialization and every access ontology that comes to mind does not
deserve that name.
</sarcasm>

Seriously: It's funny that Charlie Stross - one of my favorite Science
Fiction authors - was involved in the creation of the robots exclusion
standard.

But the "standard" is really a proprietary mess. Even basic things like
"Crawl-Delay" are extensions introduced and supported by some vendors.
Many current robots.txt libraries only check for allowed/forbidden and do not
support parsing/returning such options.

For starters, we need:

-Current extensions made official

-A means to exclude fragments of a [HTML] document for indexing

-A Noindex HTTP Header to selectively exclude content from indexing without
  bloating the robots.txt (there is an inofficial x-robots-tag supported by
  Google and Bing)

The former two would alleviate problems with the "right to be forgotten".

And possibly something to distinguish occasional Agents from recursively
crawling bots.

My current interpretation of robots.txt is that it forbids every access not
directly caused/mediated by a human.

Regards,

Michael Brunnbauer

-- ++ Michael Brunnbauer ++ netEstate GmbH ++ Geisenhausener Straße 11a ++ 81379 München ++ Tel +49 89 32 19 77 80 ++ Fax +49 89 32 19 77 89 ++ E-Mail [email protected] ++ http://www.netestate.de/ ++ ++ Sitz: München, HRB Nr.142452 (Handelsregister B München) ++ USt-IdNr. DE221033342 ++ Geschäftsführer: Michael Brunnbauer, Franz Brunnbauer ++ Prokurist: Dipl. Kfm. (Univ.) Markus Hendel

Just added <sarcasm/> to content groked by my mail post-processing agent :-)

The following could be issued via an HTTP user agent, as part of an HTTP request [1]:

Slug: RelationSubjectHint
Link: <http://kingsley.idehen.net/dataspace/person/kidehen#this> ; rel="http://example.org/action#onBehalfOf";

Enabling a server construct:

<#RelationSubjectHint> <http://example.org/action#onBehalfOf> <http://kingsley.idehen.net/dataspace/person/kidehen#this>.

Which ultimately is tested against protected resource access ACLs.


[1] http://lists.w3.org/Archives/Public/public-webpayments/2014Jul/0112.html -- issue opened in regards to Link: and HTTP requests.

--
Optimistically yours,

Kingsley Idehen 
Founder & CEO
OpenLink Software
Company Web: http://www.openlinksw.com
Personal Weblog 1: http://kidehen.blogspot.com
Personal Weblog 2: http://www.openlinksw.com/blog/~kidehen
Twitter Profile: https://twitter.com/kidehen
Google+ Profile: https://plus.google.com/+KingsleyIdehen/about
LinkedIn Profile: http://www.linkedin.com/in/kidehen
Personal WebID: http://kingsley.idehen.net/dataspace/person/kidehen#this

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

Reply via email to