I'm using Cider in Emacs, but the problems I face only occur when I've
created an uberjar and I try to run the app on the server. In the REPL I
only work with small amounts of data, whereas on the server I work with a
few million. Not sure how to get around that as I'm working from my
apartment and my client has AWS roles set up so that the database can only
be reached from the server.
I'm fine with writing everything to a log file, if that would catch
messages from background threads, though I'm curious what setups others
use? I'm a bit surprised I can't get Timbre to work the way I want to,
though I have not put much effort into configuring it to write to a log
file. In the past I've been happy to have it write to stdout and then (in
production) Supervisord would take the stdout and write it to a log file.
I'm curious what others do for logging?
On Monday, October 9, 2017 at 12:31:16 AM UTC-4, tbc++ wrote:
>
> What REPL are you using? A rather nasty side-effect of NRepl is that it
> tends to send messages from non-main-threads to weird places.
>
> On Sun, Oct 8, 2017 at 10:29 PM, <lawrence...@gmail.com <javascript:>>
> wrote:
>
>> I just re-wrote much of the code to run on the main thread. And so now I
>> can see the error message (which was about a nil value in a place where I
>> was certain there would be no nil values). Which is great. But since the
>> app does a lot of I/O, I would like to for this app to be multi-threaded.
>> But I need to figure out a way that I can see errors that happen in
>> background threads.
>>
>>
>> On Sunday, October 8, 2017 at 11:46:26 PM UTC-4, tbc++ wrote:
>>>
>>> I don't think you can catch an Object on the JVM can you? What happens
>>> if you catch Throwable?
>>>
>>> On Sun, Oct 8, 2017 at 9:43 PM, <lawrence...@gmail.com> wrote:
>>>
>>>> So, for instance, this says that 10 documents were retried:
>>>>
>>>> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 389, :retried
>>>> 10, :completed 378, :in-progress 1}}
>>>>
>>>> The only place that I call retry! is in this catch block (this function
>>>> runs in a background thread):
>>>>
>>>> (defn advance
>>>> [message]
>>>> {:pre [
>>>> (= (type message) durable_queue.Task)
>>>> ]}
>>>> (slingshot/try+
>>>> (doseq [[k v] @message]
>>>> (when-not (or
>>>> (= :how-many-rows-and-fields-from-database k)
>>>> (= :database-table k)
>>>> (= :denormalized-id k)
>>>> (= :topic k))
>>>> (transform
>>>> (get @message :denormalized-id :missing-denormalized-id)
>>>> k
>>>> v
>>>> (get @message :topic :missing-topic)
>>>> (get @message :how-many-rows-and-fields-from-database
>>>> :missing-how-many-rows-and-fields-from-database))))
>>>>
>>>> (durable/complete! message)
>>>> (catch Object o
>>>> (durable/retry! message)
>>>> (slingshot/throw+ {
>>>> :type ::advance
>>>> :error o
>>>> }
>>>> ))))
>>>>
>>>> So clearly, the code has a problem with some of the documents, and so
>>>> an error is thrown, and it is caught by the catch block. I would love to
>>>> see the error. I can not. If I put println in the catch block, the message
>>>> never shows up in the terminal. Likewise if I use Timbre.
>>>>
>>>> I would love to see the error object. My inability to see the error
>>>> objects has crippled my productivity.
>>>>
>>>> Can anyone tell me what I need to do to see these errors?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Sunday, October 8, 2017 at 11:13:10 PM UTC-4, lawrence...@gmail.com
>>>> wrote:
>>>>
>>>>> I'm trying to use a Zach Tellman's durable-queue to pass information
>>>>> from one part of my app to another. I have 3 queues. I also have a
>>>>> function
>>>>> that runs every 30 seconds, that is suppose to report some statistics for
>>>>> me:
>>>>>
>>>>> (fn []
>>>>> (future
>>>>> (slingshot/try+
>>>>> (errors/log "Resource usage: "
>>>>>
>>>>> (monitoring/show-stats-regarding-resources-used-by-this-app))
>>>>> (errors/log "Stats about
>>>>> from-mysql-to-tables-queue: " (durable/stats from-mysql-to-tables-queue))
>>>>> (errors/log "Stats about
>>>>> from-tables-to-topics-queue: " (durable/stats
>>>>> from-tables-to-topics-queue))
>>>>> (errors/log "Stats about the
>>>>> from-topics-to-persistence-queue: " (durable/stats
>>>>> from-topics-to-persistence-queue))
>>>>> (catch Object o
>>>>> (slingshot/throw+ {
>>>>> :type
>>>>> ::cycle-resource-usage-report
>>>>> :error o
>>>>> }
>>>>> )))))
>>>>>
>>>>> I think there is an error here, because I see the first (errors/log)
>>>>> but typically not anything after that. Can I rely on the fact that this
>>>>> should catch anything:
>>>>>
>>>>> (catch Object o
>>>>>
>>>>> Because it doesn't seem to work. Instead, the app runs for 8 minutes,
>>>>> then prints data about the queues just once, then never again. Assuming
>>>>> an
>>>>> error is occuring, why can't I see the error?
>>>>>
>>>>>
>>>>> {:timestamp "17-10-09 01:27:45", :level :trace, :hostname
>>>>> "ip-172-31-4-54", :message ("STARTING!!!!")}
>>>>>
>>>>> Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new
>>>>> driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically
>>>>> registered via the SPI and manual loading of the driver class is
>>>>> generally
>>>>> unnecessary.
>>>>>
>>>>>
>>>>> {:timestamp "17-10-09 01:28:16", :level :trace, :hostname
>>>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use
>>>>> (percentage/used/max-heap): (\"67%\" \"2414M\" \"3568M\")\n\nCPU usage
>>>>> (how-many-cpu's/load-average): [4 1.56]\n\nFree memory in jvm:
>>>>> [1201758848]")}
>>>>>
>>>>>
>>>>> {:timestamp "17-10-09 01:28:49", :level :trace, :hostname
>>>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use
>>>>> (percentage/used/max-heap): (\"81%\" \"2900M\" \"3568M\")\n\nCPU usage
>>>>> (how-many-cpu's/load-average): [4 2.64]\n\nFree memory in jvm:
>>>>> [699798216]")}
>>>>>
>>>>> ;; removing redundant entries
>>>>>
>>>>> {:timestamp "17-10-09 01:34:48", :level :trace, :hostname
>>>>> "UnknownHost", :message ("\n\n\n\n\nResource usage: " "Memory in use
>>>>> (percentage/used/max-heap): (\"87%\" \"3118M\" \"3568M\")\n\nCPU usage
>>>>> (how-many-cpu's/load-average): [4 3.77]\n\nFree memory in jvm:
>>>>> [471249616]")}
>>>>>
>>>>>
>>>>> {:timestamp "17-10-09 01:35:17", :level :trace, :hostname
>>>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use
>>>>> (percentage/used/max-heap): (\"87%\" \"3120M\" \"3568M\")\n\nCPU usage
>>>>> (how-many-cpu's/load-average): [4 3.79]\n\nFree memory in jvm:
>>>>> [468602192]")}
>>>>>
>>>>>
>>>>> {:timestamp "17-10-09 01:35:35", :level :trace, :hostname
>>>>> "ip-172-31-4-54", :message ("\nStats about from-mysql-to-tables-queue: "
>>>>> {"message" {:num-slabs 1, :num-active-slabs 1, :enqueued 306671, :retried
>>>>> 0, :completed 306660, :in-progress 8}})}
>>>>>
>>>>>
>>>>> {:timestamp "17-10-09 01:35:50", :level :trace, :hostname
>>>>> "ip-172-31-4-54", :message ("\nStats about from-tables-to-topics-queue: "
>>>>> {})}
>>>>>
>>>>>
>>>>> {:timestamp "17-10-09 01:35:50", :level :trace, :hostname
>>>>> "ip-172-31-4-54", :message ("\nStats about the
>>>>> from-topics-to-persistence-queue: " {})}
>>>>>
>>>>>
>>>>> {:timestamp "17-10-09 01:36:15", :level :trace, :hostname
>>>>> "ip-172-31-4-54", :message ("\n\n\n\n\nResource usage: " "Memory in use
>>>>> (percentage/used/max-heap): (\"28%\" \"1007M\" \"3568M\")\n\nCPU usage
>>>>> (how-many-cpu's/load-average): [4 4.14]\n\nFree memory in jvm:
>>>>> [1868654000]")}
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> So 8 minutes after the app starts, it print the lines with stats about
>>>>> the queues, just this one time, and then never again. I think an error
>>>>> must
>>>>> occur inside of the function that is suppose to report the queues. What
>>>>> would I have to do to see that error?
>>>>>
>>>>> My core/main function looks like this:
>>>>>
>>>>> (defn -main [& args]
>>>>>
>>>>> (slingshot/try+
>>>>>
>>>>> ;; 2017-10-04 -- see this:
>>>>> ;;
>>>>> https://stackoverflow.com/questions/28908835/ssl-peer-shut-down-incorrectly-in-java
>>>>> (System/setProperty "https.protocols" "TLSv1.1")
>>>>>
>>>>> (.addShutdownHook (Runtime/getRuntime)
>>>>> (Thread.
>>>>> #(do (println "addShutdownHook triggered. This
>>>>> app is shutting down.")
>>>>> (stop))))
>>>>>
>>>>> ;; 2017-07-12 -- stealing code from here:
>>>>> ;; https://stuartsierra.com/2015/05/27/clojure-uncaught-exceptions
>>>>> ;; Assuming require [clojure.tools.logging :as log]
>>>>> (Thread/setDefaultUncaughtExceptionHandler
>>>>> (reify Thread$UncaughtExceptionHandler
>>>>> (uncaughtException [_ thread ex]
>>>>> (println ex "Uncaught background exception on thread "
>>>>> (.getName thread) " Caught by Thread/setDefaultUncaughtExceptionHandler
>>>>> in
>>>>> core/main.")
>>>>> (throw (ex-info "/setDefaultUncaughtExceptionHandler. The app
>>>>> has died!")))))
>>>>>
>>>>> (start)
>>>>> (catch Exception e
>>>>> (println e)
>>>>> (stack/parse-exception e)
>>>>> (throw (ex-info "The app has died!")))
>>>>> (catch Object o
>>>>> (println o)
>>>>> (throw (ex-info "The app has died!")))))
>>>>>
>>>>> What else could I do to either kill the app at the moment of error or
>>>>> catch it at the top and print the error?
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Clojure" group.
>>>> To post to this group, send email to clo...@googlegroups.com
>>>> Note that posts from new members are moderated - please be patient with
>>>> your first post.
>>>> To unsubscribe from this group, send email to
>>>> clojure+u...@googlegroups.com
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/clojure?hl=en
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Clojure" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to clojure+u...@googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>>
>>> --
>>> “One of the main causes of the fall of the Roman Empire was that–lacking
>>> zero–they had no way to indicate successful termination of their C
>>> programs.”
>>> (Robert Firth)
>>>
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> <javascript:>
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com <javascript:>
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+u...@googlegroups.com <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
> --
> “One of the main causes of the fall of the Roman Empire was that–lacking
> zero–they had no way to indicate successful termination of their C
> programs.”
> (Robert Firth)
>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
