I'm going to use the "FIFIO API" term from now on... For a while I scraped Ex Libris's status webpage (http://status.exlibrisgroup.com/) to monitor the status of our catalogue/discovery layer instances and, in the event of an issue, automatically put a notice on the discovery layer to tell users what was wrong, that it was being worked on, and what other options they had in the meantime.
Eventually Ex Libris created a RESTful API with the same content so I've gratefully upgraded my code to use that instead. Deborah -----Original Message----- From: Code for Libraries [mailto:CODE4LIB@LISTS.CLIR.ORG] On Behalf Of Jason Bengtson Sent: Wednesday, 29 November 2017 8:00 a.m. To: CODE4LIB@LISTS.CLIR.ORG Subject: Re: [CODE4LIB] Anyone web scraping to benefit their library? I use web scraping sometimes to extract data from systems that lack APIs. I'm doing this to get current library job openings from our University jobs application, for instance. I use the structure of their website in a way similar to an API query, scrape the results, and extract only what I need. I jokingly call it a FIFIO API (Fine, I'll Figure It Out). Obviously, such a tool is inherently unstable, and has to be closely managed. When used with things like the jobs application, which maintain a relatively stable uri structure over time, however, it can be a pretty good tool when you have nothing else. I also used screen scraping as part of a tool I built years ago to allow authorized staff to create announcements within a special libguide that they then pushed to the EZ Proxy login page. I wrote a book chapter on that one: "Leveraging LibGuides as an EZProxy Notifications Interface." Innovative Libguides Applications: Real World Examples. New York: Rowman & Littlefield, 2016 Best regards, *Jason Bengtson* *http://www.jasonbengtson.com/ <http://www.jasonbengtson.com/>* On Tue, Nov 28, 2017 at 12:24 PM, Brad Coffield <bcoffield.libr...@gmail.com > wrote: > I think there's likely a lot of possibilities out there and was hoping > to hear examples of web scraping for libraries. Your example might > just inspire me or another reader to do something similar. At the very > least, the ideas will be interesting! > > Brad > > > -- > Brad Coffield, MLIS > Assistant Information and Web Services Librarian Saint Francis > University > 814-472-3315 > bcoffi...@francis.edu > ________________________________ P Please consider the environment before you print this email. "The contents of this e-mail (including any attachments) may be confidential and/or subject to copyright. Any unauthorised use, distribution, or copying of the contents is expressly prohibited. If you have received this e-mail in error, please advise the sender by return e-mail or telephone and then delete this e-mail together with all attachments from your system."
