Konstantinov, yes, perhaps, though I can't think where. This is a small app, about 950 lines of code. There is a limited number of places where I can make such a mistake. I do aggregate a lot of data into an atom, and I'm sure there is a lot of contention around the atom, but the Socket timeout errors are surely some weird AWS or ElasticSearch thing.
On Thursday, October 5, 2017 at 2:12:42 PM UTC-4, Lubomir Konstantinov wrote: > > Holding to a head somewhere perhaps? > > On Thursday, 5 October 2017 19:32:11 UTC+3, lawrence...@gmail.com wrote: >> >> One last thing, I should mention, if I go through and add " LIMIT 100 " >> to all the SQL queries, everything works great. That is, when dealing with >> a few hundred documents, the app seems to work perfectly, and there are no >> errors. It's only when I try to work with a few million documents that >> things fall apart. I suspect some kind of contention arises... somewhere. >> Possibly in several locations. But I'm not yet sure what the problem is. >> >> >> >> On Wednesday, October 4, 2017 at 12:49:12 AM UTC-4, lawrence...@gmail.com >> wrote: >>> >>> This is probably a stupid question, but is there an obvious way to get >>> an error message out of Elastisch? I had an app that was working with >>> MongoDB and I was then told I had to use ElasticSearch instead (something >>> about only using AWS for everything) so now I'm trying to get an error >>> message, because my code doesn't seem to work. I read through here without >>> seeing anything obvious: >>> >>> http://clojureelasticsearch.info/articles/getting_started.htm >>> >>> I rewrote my MongoDB function, so it should work with Elastisch: >>> >>> (defn push-item-to-persistence >>> [item db] >>> (let [ >>> denormalized-id (get-in item [:denormalized-id] :no-id) >>> item (assoc item :updated-at (temporal/current-time-as-datetime)) >>> item (assoc item :permanent-holding-id-for-item-instances >>> (java.util.UUID/randomUUID)) >>> item (assoc item :instance-id-for-this-one-item >>> (java.util.UUID/randomUUID)) >>> item (assoc item :item-type :deduplication) >>> ] >>> (if (= denormalized-id :no-id) >>> (slingshot/throw+ { >>> :type >>> ::no-denormalized-id-in-push-item-into-database >>> :item item >>> }) >>> (slingshot/try+ >>> (put conn index mapping-type id document) >>> (esd/put db "facts-over-time" "deduplicaton" (str >>> denormalized-id) item) >>> (println " done with put in push-item-to-persistence ") >>> (catch Object o >>> (slingshot/throw+ { >>> :type ::push-item-to-persistence >>> :error o >>> :item item >>> :db db >>> } >>> )))))) >>> >>> >>> It doesn't seem that any documents are getting into ElasticSearch. I was >>> hoping to the (throw) would reveal to me some useful debugging information, >>> but that doesn't seem to happen. >>> >>> The connection appears to be valid, as I can do this: >>> >>> (let [conn (persistence/multi-thread-start config) >>> res (esd/search conn "facts-over-time" >>> "deduplication" :query {:match_all {}}) >>> n (esrsp/total-hits res) >>> hits (esrsp/hits-from res)] >>> (clojure.pprint/pprint res)) >>> >>> and I get: >>> >>> {:took 1, >>> :timed_out false, >>> :_shards {:total 5, :successful 5, :failed 0}, >>> :hits {:total 0, :max_score nil, :hits []}} >>> >>> So the connection is there. But no records are. >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated - please be patient with your first post. To unsubscribe from this group, send email to clojure+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/clojure?hl=en --- You received this message because you are subscribed to the Google Groups "Clojure" group. To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
