Can you clarify, do you mean:- Only allow GoogleBot? Only crawl JSPWiki? Both?
On 17 February 2016 at 01:39, Jason Morris <jason.mor...@sydney.edu.au> wrote: > Hi All, > > What would be the proper config of robots.txt to allow only Google to > crawl a JSPWiki installation and nothing else? > > Cheers, > Jason > > Jason C. Morris | PhD Candidate > Department of Environmental Sciences | Faculty of Agriculture and > Environment > THE UNIVERSITY OF SYDNEY, NSW, 2006 > phone: +61 02 8627 1152 > >
