Hi...
I have query regarding the clojure and neo4j..
I have started the program to load the dataset and also allocated 4G ram to
it.
But after processing around 272000 records, the process hangs and nothing
happens. If i try to stop the process then, and copy the data in neo4j
folder i get this error :
'neostore' does not contain a store version, please ensure that the
original database was shut down in a clean state.
Can you please advice what can be done on that and how should i proceed
further?
Many Thanks,
Kind Regards
Himakshi
On Sat, Dec 7, 2013 at 1:15 AM, Joseph Guhlin <joseph.guh...@gmail.com>wrote:
> Glad it worked, if you have any further questions feel free to ask. I'm
> using it extensively and it and Clojure seem to be a perfect match these
> days, especially on very large datasets.
>
> --Joseph
>
>
> On Friday, December 6, 2013 12:56:40 AM UTC-6, Himakshi Mangal wrote:
>>
>> Hi Joseph Guhlin,
>>
>> Thanks your idea helped and i could send some sample data to my neo4j
>> database.
>>
>> Thank you very much... :)
>>
>>
>> On Monday, December 2, 2013 3:41:53 PM UTC+5:30, Himakshi Mangal wrote:
>>>
>>> Hi...
>>>
>>>
>>> I am using clojure to import dbpedia data into neo4j.
>>>
>>> Here's the code:
>>> (ns opal.dbpedia
>>> (:use [clojure.tools.logging :only [log]])
>>> (:require [clojure.java.io :as io])
>>> (:import [uk.ac.manchester.cs.owl.owlapi.turtle.parser TurtleParser]
>>> [org.neo4j.unsafe.batchinsert BatchInserters]
>>> [org.neo4j.graphdb DynamicRelationshipType]))
>>>
>>> ;; PARSING METHODS
>>>
>>> (defn get-next-tuple
>>> [parser]
>>> (let [last-item (atom nil)
>>> tuple (atom [])]
>>> (while (and (not= "." @last-item)
>>> (not= "" @last-item))
>>> (reset! last-item
>>> (-> parser
>>> (.getNextToken)
>>> (.toString)))
>>> (swap! tuple conj @last-item))
>>> (when-not (empty? (first @tuple)) ; .getNextToken returns "" once
>>> you are out of data
>>> @tuple)))
>>>
>>> (defn seq-of-parser
>>> [parser]
>>> (if-let [next-tuple (get-next-tuple parser)]
>>> (lazy-cat [next-tuple]
>>> (seq-of-parser parser))))
>>>
>>> (defn parse-file
>>> [filename]
>>> (seq-of-parser
>>> (TurtleParser.
>>> (io/input-stream filename))))
>>>
>>> ;; BATCH UPSERT METHODS
>>>
>>> (def id-map (atom nil))
>>> (defn insert-resource-node!
>>> [inserter res]
>>> (if-let [id (get @id-map res)]
>>> ; If the resource has aleady been added, just return the id.
>>> id
>>> ; Otherwise, add the node for the node, and remember its id for
>>> later.
>>> (let [id (.createNode inserter {"resource" res})]
>>> (swap! id-map #(assoc! % res id))
>>> id)))
>>>
>>> (defn connect-resource-nodes!
>>> [inserter node1 node2 label]
>>> (let [relationship (DynamicRelationshipType/withName label)]
>>> (.createRelationship inserter node1 node2 relationship nil)))
>>>
>>> (defn insert-tuple!
>>> [inserter tuple]
>>> ; Get the resource and label names out of the tuple.
>>> (let [[resource-1 label resource-2 & _ ] tuple
>>> ; Upsert the resource nodes.
>>> node-1 (insert-resource-node! inserter resource-1)
>>> node-2 (insert-resource-node! inserter resource-2)]
>>> ; Connect the nodes with an edge.
>>> (connect-resource-nodes! inserter node-1 node-2 label)))
>>>
>>> (defn -main [graph-path & files]
>>> (let [inserter (BatchInserters/inserter graph-path)]
>>> (doseq [file files]
>>> (log :debug (str "Loading file: " file))
>>> (let [c (atom 0)]
>>> (doseq [tuple (parse-file file)]
>>> (if (= (mod @c 10000) 0)
>>> (log :debug (str file ": " @c)))
>>> (swap! c inc)
>>> (insert-tuple! inserter tuple))))
>>> (log :debug "Loading complete.")
>>> (log :debug "Shutting down.")
>>> (.shutdown inserter)
>>> (log :debug "Shutdown complete!")))
>>>
>>> I am getting the following errors:
>>>
>>> IllegalAccessError Transient used by non-owner thread clojure.lang.
>>> PersistentArrayMap$TransientArrayMap.ensureEditable
>>> (PersistentArrayMap.java:449)
>>>
>>> &&
>>>
>>>
>>> IllegalArgumentException No matching method found: createNode for class
>>> org.neo4j.unsafe.batchinsert.BatchInserterImpl clojure.lang.Reflector.
>>> invokeMatchingMethod
>>>
>>>
>>> Can anyone please help me in this.. Am doing something wrong or am i
>>> missing something.. I am completely new to clojure. Is there a working
>>> example for this?
>>>
>>>
>>> Please help..
>>>
>>> Thanks
>>>
>>>
>>> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "Clojure" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/clojure/HpI6wKHb8Pc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
