I don't recommend using java's built in HTTP retrieval (by passing
java.net.URL object to enlive html-resource function).
Not only is it significantly slower then using clj-http (which uses
apache-http client under the hood), but it's also unreliable
when issuing more parallel requests.
Current enlive library supports plug-able parsers, the default one is
TagSoup, but you can switch it very easily for example
for JSoup by setting *parser* dynamic var.
You can have a look at one of my little projects where i used enlive for
html scraping
here<https://github.com/janherich/lazada-quest/blob/master/src/lazada_quest/scrapper.clj>
,
in this case, i used clj-http as
http client:
(ns lazada-quest.scrapper
(:require [clojure.string :as string]
[clj-http.client :as client]
[net.cgrand.enlive-html :as html]))
(defn fetch-url
"Given some url string, fetch html content of the resource served under url
adress and return
it in the form of enlive nodes"
[url]
(html/html-resource (:body (client/get url {:as :stream}))))
It would be straightforward to replace use of clj-http with http-kit
synchronous api, or asynchronous api with some changes
Dňa nedeľa, 12. januára 2014 0:24:48 UTC+1 Dave Tenny napísal(-a):
>
> I'm just playing around with tool kits to retrieve and parse html from web
> pages and files that I already have on disk (such as JDK API documentation).
>
> Based on too little time, it looks like [http-kit "2.1.16"] will retrieve
> but not parse html, and [enlive "1.1.5"] will retrieve AND parse html.
>
> Or is there a whole built-in parse capability I'm missing in http-kiit?
>
> Also, http-kit doesn't seem to want to retrieve content from a file:///
> url, whereas enlive is happy with both local and remote content.
>
> I'm just messing around, I wanted to have some REPL javadoc logic that
> didn't fire up a browser or use the swing app (whose fonts are unreadable
> for me, and half a day spent trying to change it was not fruitful).
>
> Any tips or suggestions? Just don't want to make sure I'm missing obvious
> things.
>
> Thanks!
>
>
>
>
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to [email protected]
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
