Re: bots in closure

[Daniel Compton]

Hi Frank

https://github.com/wit-ai/duckling_old is a text parsing library that
returns structured data. You might find that useful?

On Tue, Jul 11, 2017 at 1:10 PM Alex Walker  wrote:

> Hi Frank,
>
> Chatbot is a bit of a loaded term, so could you please give a bit of
> clarification?
>
>1. Are you expecting natural language (spoken or text), or
>rigid/regex-based commands for user input?
>   - If NL,
>  - do you want to leverage an existing service for intent
>  classification and entity extraction, like Alexa, Wit.ai, Luis, 
> Api.ai, etc
>  - or are you looking to build your own models using the
>  Clojure/JVM ecosystem?
>   - Else,
>  - are you after a hubot equivalent in clojure?
>   2. Do you want your bot to talk to various chat platforms (slack,
>facebook, etc) or do you have a specific one in mind?
>
>
> On Monday, July 10, 2017 at 11:06:45 AM UTC-5, ureal frank wrote:
>>
>> Hi,
>>
>> Is anyone building chatbots in Clojure?
>> OR can someone point me something to start with?
>>
>> Clojure newbie here :)
>>
>> thanks 🙏
>> Frank
>>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [ANN] r4f-pro: IDE for visual ontology-based, rule-based and algorithm-based software development

[ru]

Hi Didier,

I believe that main use case is fast prototyping of new ideas. You model 
new concepts, classes, relations using native Protege capabilities and 
implement processes in this new area using rules and algorithms depending 
of what suits better. 
Important thing here is visual representation of all this things. It is 
useful for presentation, reasoning, etc. In general, people think better 
when visually represent subject (arguably). 

One use case is here: https://github.com/rururu/flights-pro

Sincerely,
  Ru

вторник, 11 июля 2017 г., 9:38:16 UTC+3 пользователь Didier написал:
>
> I feel like I'm missing out on this. It sounds very cool, but I'm not 
> sureI understand the use cases whereI should reach for it. Can you speak of 
> what kind of use case you can use rete and r4j for?

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[Justin Smith]

My first suspicion would be that by the time you access the second element,
you have exited the context of your database transaction, so there's no
data stream available to get it from. Lazyness doesn't tend to mix well
with stateful resources and contexts.

On Mon, Jul 10, 2017 at 9:45 PM  wrote:

> Okay, that was a deadend. After going through line by line, I could pretty
> well rule out any kind of Exception or Error. The first row from the
> database seems to go through all of the functions perfectly, without any
> errors. But the second row never gets called.
>
> Which takes me back to where I started. Why can't I give a lazy-seq to a
> Manifold Stream and simply have a function map over the stream?
>
>
> On Monday, July 10, 2017 at 11:25:43 PM UTC-4, lawrence...@gmail.com
> wrote:
>>
>>
>> Once again, my lack of knowledge of Java trips me up. Manifold relies on
>> Dirigiste, which relies on Java's Executor Service. I see a bit here:
>>
>> http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html
>>
>> Nurkiewicz writes:
>> "I got bitten by that too many times: it won't print *anything*. No sign
>> of java.lang.ArithmeticException: / by zero, nothing. Thread pool just
>> swallows this exception, as if it never happened. If it was a good'ol
>> java.lang.Thread created from scratch, UncaughtExceptionHandler
>> 
>>  could
>> work. "
>>
>> I suspect I'm facing something like that.
>>
>>
>>
>> On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com
>> wrote:
>>>
>>> By the way, this code works fine if I go into a (doseq) a level above
>>> enqueue, and then put each individual row onto the stream. Then the code
>>> loops over all of the rows. But that seems to defeat the whole point of
>>> using something like Manifold. I want to be able to put the whole lazy-seq
>>> on the stream. That is supposed to work, yes?
>>>
>>>
>>> On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com
>>> wrote:

 I'm using Zach Tellman's excellent Manifold library, though I admit I
 don't fully understand it.

 My code queries a MySQL database and then needs to do some processing
 on each row retrieved. I copy-and-pasted some code from the documentation
 for Manifold:


 ;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200
 threads.
 ;; query_database/start will pull data from the database and dump it
 onto a
 ;; stream below, at which point each row should be assigned to one of
 the rows
 ;; on our thread pool.
 (def executor (me/fixed-thread-executor 200))


 (defn enqueue
   [sequence-from-database]
   (slingshot/try+
(println "the type of the object from the database: " (type
 sequence-from-database))
(->> (ms/->source sequence-from-database)
 (ms/onto executor)
 (ms/map api/query))
(catch Object o
  (println " message queue is not happy about the message we were
 given")
  (errors/error o "" " we tried to put something on the message
 queue, but we got an error "

 The line where I print out the type assures that I'm dealing with a
 LazySeq.

 The code prints out the type and the first row:


 the type of the object from the database:  clojure.lang.LazySeq

 in query_api/query  {:profile_id 2, :profile_name Mike Shaw Automotive
 Group, :headquarters_addr1 90 Madison St., :headquarters_city Denver,
 :headquarters_state_code CO, :headquarters_country_code US, :url
 mikeshawauto.com}

 and then it stops. I assume there must be an Exception or Error
 happening, but I can't find it. I've added as many general Catch clauses as
 I could:


 (defn query
   [row & args]

   (println " in query_api/query " row)

   (let [config (if args
  (first args))
 ]
 ;; 2017-03-30 -- the API is overwhelmed and all I get is Socket
 Timeout errors
 (Thread/sleep 300)

 (slingshot/try+
  (call-api row)
  (catch Object o
(println " error : " o)

;;(errors/error o row " we tried to call-api, but we got this
 error ")


)

  (catch Error e
(println " there Error: " e))
  )))

 So if I do:

 java  -jar scan-database-find-similar-items-standalone.jar


 The code runs till it prints out the first row from the database, and
 then it stops. Nothing else happens. There are no error messages.

 What did I miss?







 --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are mod

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[lawrence . krubner]

Justin, thanks. I'd like to ask a follow up question. To be clear, if I go 
into (doseq) then this works fine: 

(defn start []
  (let [
config (get-config)
mysql-db {
  :dbtype "mysql"
  :dbname (get-in  config [:database :config :connection 
:database])
  :user (get-in  config [:database :config :connection 
:user] )
  :password (get-in  config [:database :config :connection 
:password] )
  :host (get-in  config [:database :config :connection 
:host])
  }
data (fetch mysql-db)
]
(slingshot/try+
 (doseq [row data]
   (queue/enqueue row))
 (catch Object o
   (errors/error o ""  "error in query_database/start: ")


but if I do this instead:

;;;  (doseq [row data]
   (queue/enqueue data))
   
and then enqueue does this: 

   (->> (ms/->source data)
(ms/onto executor)
(ms/map api/query))


The first row comes through fine, but then the second row is never 
processed. 

So are you suggesting that simply passing "data" from one function to the 
next is enough to lose the database context? But only after the first row 
has been pulled? 






On Tuesday, July 11, 2017 at 1:44:15 PM UTC-4, Justin Smith wrote:
>
> My first suspicion would be that by the time you access the second 
> element, you have exited the context of your database transaction, so 
> there's no data stream available to get it from. Lazyness doesn't tend to 
> mix well with stateful resources and contexts.
>
> On Mon, Jul 10, 2017 at 9:45 PM > 
> wrote:
>
>> Okay, that was a deadend. After going through line by line, I could 
>> pretty well rule out any kind of Exception or Error. The first row from the 
>> database seems to go through all of the functions perfectly, without any 
>> errors. But the second row never gets called. 
>>
>> Which takes me back to where I started. Why can't I give a lazy-seq to a 
>> Manifold Stream and simply have a function map over the stream? 
>>
>>
>> On Monday, July 10, 2017 at 11:25:43 PM UTC-4, lawrence...@gmail.com 
>> wrote:
>>>
>>>
>>> Once again, my lack of knowledge of Java trips me up. Manifold relies on 
>>> Dirigiste, which relies on Java's Executor Service. I see a bit here:
>>>
>>> http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html
>>>
>>> Nurkiewicz writes:
>>> "I got bitten by that too many times: it won't print *anything*. No 
>>> sign of java.lang.ArithmeticException: / by zero, nothing. Thread pool 
>>> just swallows this exception, as if it never happened. If it was a good'ol 
>>> java.lang.Thread created from scratch, UncaughtExceptionHandler 
>>> 
>>>  could 
>>> work. "
>>>
>>> I suspect I'm facing something like that. 
>>>
>>>
>>>
>>> On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com 
>>> wrote:

 By the way, this code works fine if I go into a (doseq) a level above 
 enqueue, and then put each individual row onto the stream. Then the code 
 loops over all of the rows. But that seems to defeat the whole point of 
 using something like Manifold. I want to be able to put the whole lazy-seq 
 on the stream. That is supposed to work, yes? 


 On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com 
 wrote:
>
> I'm using Zach Tellman's excellent Manifold library, though I admit I 
> don't fully understand it. 
>
> My code queries a MySQL database and then needs to do some processing 
> on each row retrieved. I copy-and-pasted some code from the documentation 
> for Manifold: 
>
>
> ;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200 
> threads.
> ;; query_database/start will pull data from the database and dump it 
> onto a
> ;; stream below, at which point each row should be assigned to one of 
> the rows
> ;; on our thread pool. 
> (def executor (me/fixed-thread-executor 200))
>
>
> (defn enqueue
>   [sequence-from-database]
>   (slingshot/try+
>(println "the type of the object from the database: " (type 
> sequence-from-database))
>(->> (ms/->source sequence-from-database)
> (ms/onto executor)
> (ms/map api/query))
>(catch Object o
>  (println " message queue is not happy about the message we were 
> given")
>  (errors/error o "" " we tried to put something on the message 
> queue, but we got an error "
>
> The line where I print out the type assures that I'm dealing with a 
> LazySeq.
>
> The code prints out the type and the first row: 
>
>
> the type of the object from the database:  clojure.lang.LazySeq
>  
> in query_api/query  {:profile_id 2, :profile_name Mike Shaw Automotive 
> Group, :headquarters_addr1 90 

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[Justin Smith]

Oh - this appears to be a separate issue that isn't just about leaving the
context of the db transaction, I don't know enough about those parts of
manifold to address the issue, sorry.

On Tue, Jul 11, 2017 at 1:18 PM  wrote:

> Justin, thanks. I'd like to ask a follow up question. To be clear, if I go
> into (doseq) then this works fine:
>
> (defn start []
>   (let [
> config (get-config)
> mysql-db {
>   :dbtype "mysql"
>   :dbname (get-in  config [:database :config :connection
> :database])
>   :user (get-in  config [:database :config :connection
> :user] )
>   :password (get-in  config [:database :config :connection
> :password] )
>   :host (get-in  config [:database :config :connection
> :host])
>   }
> data (fetch mysql-db)
> ]
> (slingshot/try+
>  (doseq [row data]
>(queue/enqueue row))
>  (catch Object o
>(errors/error o ""  "error in query_database/start: ")
>
>
> but if I do this instead:
>
> ;;;  (doseq [row data]
>(queue/enqueue data))
>
> and then enqueue does this:
>
>(->> (ms/->source data)
> (ms/onto executor)
> (ms/map api/query))
>
>
> The first row comes through fine, but then the second row is never
> processed.
>
> So are you suggesting that simply passing "data" from one function to the
> next is enough to lose the database context? But only after the first row
> has been pulled?
>
>
>
>
>
>
> On Tuesday, July 11, 2017 at 1:44:15 PM UTC-4, Justin Smith wrote:
>
>> My first suspicion would be that by the time you access the second
>> element, you have exited the context of your database transaction, so
>> there's no data stream available to get it from. Lazyness doesn't tend to
>> mix well with stateful resources and contexts.
>>
>> On Mon, Jul 10, 2017 at 9:45 PM  wrote:
>>
> Okay, that was a deadend. After going through line by line, I could pretty
>>> well rule out any kind of Exception or Error. The first row from the
>>> database seems to go through all of the functions perfectly, without any
>>> errors. But the second row never gets called.
>>>
>>> Which takes me back to where I started. Why can't I give a lazy-seq to a
>>> Manifold Stream and simply have a function map over the stream?
>>>
>>>
>>> On Monday, July 10, 2017 at 11:25:43 PM UTC-4, lawrence...@gmail.com
>>> wrote:


 Once again, my lack of knowledge of Java trips me up. Manifold relies
 on Dirigiste, which relies on Java's Executor Service. I see a bit here:


 http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html

 Nurkiewicz writes:
 "I got bitten by that too many times: it won't print *anything*. No
 sign of java.lang.ArithmeticException: / by zero, nothing. Thread pool
 just swallows this exception, as if it never happened. If it was a good'ol
 java.lang.Thread created from scratch, UncaughtExceptionHandler
 
  could
 work. "

 I suspect I'm facing something like that.



 On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com
 wrote:
>
> By the way, this code works fine if I go into a (doseq) a level above
> enqueue, and then put each individual row onto the stream. Then the code
> loops over all of the rows. But that seems to defeat the whole point of
> using something like Manifold. I want to be able to put the whole lazy-seq
> on the stream. That is supposed to work, yes?
>
>
> On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com
> wrote:
>>
>> I'm using Zach Tellman's excellent Manifold library, though I admit I
>> don't fully understand it.
>>
>> My code queries a MySQL database and then needs to do some processing
>> on each row retrieved. I copy-and-pasted some code from the documentation
>> for Manifold:
>>
>>
>> ;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200
>> threads.
>> ;; query_database/start will pull data from the database and dump it
>> onto a
>> ;; stream below, at which point each row should be assigned to one of
>> the rows
>> ;; on our thread pool.
>> (def executor (me/fixed-thread-executor 200))
>>
>>
>> (defn enqueue
>>   [sequence-from-database]
>>   (slingshot/try+
>>(println "the type of the object from the database: " (type
>> sequence-from-database))
>>(->> (ms/->source sequence-from-database)
>> (ms/onto executor)
>> (ms/map api/query))
>>(catch Object o
>>  (println " message queue is not happy about the message we were
>> given")
>>  (errors/error o "" " we tried to put something on the message
>> queue, but we got an error "
>>
>> The

Re: [ClojureScript] [ANN] Orchestra 2017.07.04-1 (now supporting ClojureScript)

[Daniel Compton]

> It might be worth including a discussion about when to use this library,
and perhaps indicating that using it might not be a best-practice.

I'm using it to instrument a running development system as I'm working with
it, via an API, REPL, and tests. I've found it handy in all three places
for my :ret specs to be checked when the function runs, and quickly
detecting problems in my spec or function. I can see how :ret and :fn are
invaluable for generative tests, but in my experience it is also very
useful to check them in other places. As far as I can tell, the only
downside to checking :fn and :ret specs while in development is that it
takes marginally more CPU to check them. Maybe I'm missing something though?

On Tue, Jul 11, 2017 at 7:40 AM Didier  wrote:

> I find it funny that Clojure strongly believes that static types aren't
> worth the effort in most cases, but somehow the effort of adding generative
> testing is.
>
> I think it's great to encourage people to use generative testing, but I'd
> rather it be à la carte, like most other things in Clojure. Orchestra
> allows that. If you want your :fn and :ret spec validated only on your
> manual tests, or on your regression tests, it lets you do that.
>
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Deserialization "gadget chain" in clojure

[Ian Haken]

Dear Clojure community,

First off, apologies for directing this at the general clojure mailing 
list. I was looking for a better destination, but I couldn't find any 
obvious person or private mailing list to direct this to; hopefully from 
here this can get in front of anyone who may be interested.

I recently identified a means to exploit an application performing unsafe 
deserialization by utilizing a "gadget chain" contained entirely in Clojure 
classes. What this means is that any application which is deserializing 
untrusted data and has Clojure on the classpath (whether or not it is 
actually using Clojure) is subject to a remote code execution exploit. Some 
more information on this form of vulnerability is available on the OWASP 
wiki , 
which also has a number of useful links for deeper discussion.

Details of the specific gadget chain I found can be seen here 
.

This does not represent a "security vulnerability" in Clojure, and I do not 
necessarily believe that any action needs to be taken. If an application is 
subject to exploit via this gadget chain, the vulnerability is with 
application deserializing untrusted data. However, when deserialization 
vulnerabilities became a hot topic a couple of years ago with the discovery 
of a gadget chain in apache-commons 
,
 
that project received a lot of attention and some made the argument that 
the projects which support exploitable gadget chains should apply some form 
of mitigation. For this reason, I wanted to give maintainers of this 
project a heads-up in case there was any particular action you want to take.

If you have any questions or if there's anything I can clarify, please 
don't hesitate to reach out to me.
Ian Haken
Twitter: @ianhaken

-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

RE: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[Sean Corfield]

What is the ‘fetch’ function here?

Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
An Architect's View -- http://corfield.org/

"If you're not annoying somebody, you're not really alive."
-- Margaret Atwood

From: lawrence.krub...@gmail.com
Sent: Tuesday, July 11, 2017 1:18 PM
To: Clojure
Subject: Re: I can only get the first item of a lazyseq via a Manifold stream, 
and I can't get/find an Exception

Justin, thanks. I'd like to ask a follow up question. To be clear, if I go into 
(doseq) then this works fine:

(defn start []
  (let [
config (get-config)
mysql-db {
  :dbtype "mysql"
  :dbname (get-in  config [:database :config :connection 
:database])
  :user (get-in  config [:database :config :connection :user] )
  :password (get-in  config [:database :config :connection 
:password] )
  :host (get-in  config [:database :config :connection :host])
  }
data (fetch mysql-db)
]
(slingshot/try+
 (doseq [row data]
   (queue/enqueue row))
 (catch Object o
   (errors/error o ""  "error in query_database/start: ")


but if I do this instead:

;;;  (doseq [row data]
   (queue/enqueue data))

and then enqueue does this:

   (->> (ms/->source data)
(ms/onto executor)
(ms/map api/query))


The first row comes through fine, but then the second row is never processed.

So are you suggesting that simply passing "data" from one function to the next 
is enough to lose the database context? But only after the first row has been 
pulled?






On Tuesday, July 11, 2017 at 1:44:15 PM UTC-4, Justin Smith wrote:
My first suspicion would be that by the time you access the second element, you 
have exited the context of your database transaction, so there's no data stream 
available to get it from. Lazyness doesn't tend to mix well with stateful 
resources and contexts.

On Mon, Jul 10, 2017 at 9:45 PM > wrote:
Okay, that was a deadend. After going through line by line, I could pretty well 
rule out any kind of Exception or Error. The first row from the database seems 
to go through all of the functions perfectly, without any errors. But the 
second row never gets called.

Which takes me back to where I started. Why can't I give a lazy-seq to a 
Manifold Stream and simply have a function map over the stream?


On Monday, July 10, 2017 at 11:25:43 PM UTC-4, lawrence...@gmail.com wrote:

Once again, my lack of knowledge of Java trips me up. Manifold relies on 
Dirigiste, which relies on Java's Executor Service. I see a bit here:

http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html

Nurkiewicz writes:
"I got bitten by that too many times: it won't print anything. No sign of 
java.lang.ArithmeticException: / by zero, nothing. Thread pool just swallows 
this exception, as if it never happened. If it was a good'ol java.lang.Thread 
created from scratch, 
UncaughtExceptionHandler
 could work. "

I suspect I'm facing something like that.



On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com wrote:
By the way, this code works fine if I go into a (doseq) a level above enqueue, 
and then put each individual row onto the stream. Then the code loops over all 
of the rows. But that seems to defeat the whole point of using something like 
Manifold. I want to be able to put the whole lazy-seq on the stream. That is 
supposed to work, yes?


On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com wrote:
I'm using Zach Tellman's excellent Manifold library, though I admit I don't 
fully understand it.

My code queries a MySQL database and then needs to do some processing on each 
row retrieved. I copy-and-pasted some code from the documentation for Manifold:


;; 2017-07-10 -- we want a thread pool. I'm arbitrarily choosing 200 threads.
;; query_database/start will pull data from the database and dump it onto a
;; stream below, at which point each row should be assigned to one of the rows
;; on our thread pool.
(def executor (me/fixed-thread-executor 200))


(defn enqueue
  [sequence-from-database]
  (slingshot/try+
   (println "the type of the object from the database: " (type 
sequence-from-database))
   (->> (ms/->source sequence-from-database)
(ms/onto executor)
(ms/map api/query))
   (catch Object o
 (println " message queue is not happy about the message we were given")
 (errors/error o "" " we tried to put something on the message queue, but 
we got an error "

The line where I print out the type assures that I'm dealing with a LazySeq.

The code prints out the type and the first row:


the type of the object from the database:  clojure.lang.LazySeq

in query_api/query  {:profile_id 2, :profile_name Mike Shaw Automotive Group, 
:head

Re: I can only get the first item of a lazyseq via a Manifold stream, and I can't get/find an Exception

[lawrence . krubner]

Sean, thanks for asking. The function is so basic that I didn't include it, 
but here it is:


(defn fetch [mysql-db]
  (slingshot/try+
   (jdbc/query mysql-db
   ["  SELECT p.id as profile_id,  p.name as profile_name, 
p.headquarters_addr1,  p.headquarters_city,  p.headquarters_state_code, 
headquarters_country_code, cw.url  FROM  company_profile p LEFT JOIN 
company_website cw ON p.id = cw.company_profile_id WHERE p.name is not null 
and p.name != ''  "])
   (catch Object o
 (errors/error o ""  " query to database had a problem "




On Wednesday, July 12, 2017 at 2:09:32 AM UTC-4, Sean Corfield wrote:
>
> What is the ‘fetch’ function here?
>
>  
>
> Sean Corfield -- (970) FOR-SEAN -- (904) 302-SEAN
> An Architect's View -- http://corfield.org/
>
> "If you're not annoying somebody, you're not really alive."
> -- Margaret Atwood
>
>  
>
> *From: *lawrence...@gmail.com 
> *Sent: *Tuesday, July 11, 2017 1:18 PM
> *To: *Clojure 
> *Subject: *Re: I can only get the first item of a lazyseq via a Manifold 
> stream, and I can't get/find an Exception
>
>  
>
> Justin, thanks. I'd like to ask a follow up question. To be clear, if I go 
> into (doseq) then this works fine:  
>
>  
>
> (defn start []
>
>   (let [
>
> config (get-config)
>
> mysql-db {
>
>   :dbtype "mysql"
>
>   :dbname (get-in  config [:database :config :connection 
> :database])
>
>   :user (get-in  config [:database :config :connection 
> :user] )
>
>   :password (get-in  config [:database :config :connection 
> :password] )
>
>   :host (get-in  config [:database :config :connection 
> :host])
>
>   }
>
> data (fetch mysql-db)
>
> ]
>
> (slingshot/try+
>
>  (doseq [row data]
>
>(queue/enqueue row))
>
>  (catch Object o
>
>(errors/error o ""  "error in query_database/start: ")
>
>  
>
>  
>
> but if I do this instead:
>
>  
>
> ;;;  (doseq [row data]
>
>(queue/enqueue data))
>
>
>
> and then enqueue does this: 
>
>  
>
>(->> (ms/->source data)
>
> (ms/onto executor)
>
> (ms/map api/query))
>
>  
>
>  
>
> The first row comes through fine, but then the second row is never 
> processed. 
>
>  
>
> So are you suggesting that simply passing "data" from one function to the 
> next is enough to lose the database context? But only after the first row 
> has been pulled? 
>
>  
>
>  
>
>  
>
>  
>
>  
>
>
> On Tuesday, July 11, 2017 at 1:44:15 PM UTC-4, Justin Smith wrote: 
>
> My first suspicion would be that by the time you access the second 
> element, you have exited the context of your database transaction, so 
> there's no data stream available to get it from. Lazyness doesn't tend to 
> mix well with stateful resources and contexts.
>
>  
>
> On Mon, Jul 10, 2017 at 9:45 PM  wrote:
>
> Okay, that was a deadend. After going through line by line, I could pretty 
> well rule out any kind of Exception or Error. The first row from the 
> database seems to go through all of the functions perfectly, without any 
> errors. But the second row never gets called.  
>
>  
>
> Which takes me back to where I started. Why can't I give a lazy-seq to a 
> Manifold Stream and simply have a function map over the stream? 
>
>
>
> On Monday, July 10, 2017 at 11:25:43 PM UTC-4, lawrence...@gmail.com 
> wrote: 
>
>  
>
> Once again, my lack of knowledge of Java trips me up. Manifold relies on 
> Dirigiste, which relies on Java's Executor Service. I see a bit here:
>
>  
>
> http://www.nurkiewicz.com/2014/11/executorservice-10-tips-and-tricks.html
>
>  
>
> Nurkiewicz writes:
>
> "I got bitten by that too many times: it won't print *anything*. No sign 
> of java.lang.ArithmeticException: / by zero, nothing. Thread pool just 
> swallows this exception, as if it never happened. If it was a good'ol 
> java.lang.Thread created from scratch, UncaughtExceptionHandler 
> 
>  could 
> work. "
>
>  
>
> I suspect I'm facing something like that. 
>
>  
>
>
>
> On Monday, July 10, 2017 at 8:28:03 PM UTC-4, lawrence...@gmail.com 
> wrote: 
>
> By the way, this code works fine if I go into a (doseq) a level above 
> enqueue, and then put each individual row onto the stream. Then the code 
> loops over all of the rows. But that seems to defeat the whole point of 
> using something like Manifold. I want to be able to put the whole lazy-seq 
> on the stream. That is supposed to work, yes?  
>
>
>
> On Monday, July 10, 2017 at 8:18:07 PM UTC-4, lawrence...@gmail.com 
> wrote: 
>
> I'm using Zach Tellman's excellent Manifold library, though I admit I 
> don't fully understand it.  
>
>  
>
> My code queries a MySQL database and then needs to do some processing on 
> each row retrieved. I copy-and-pasted some code from the documentation for 
> Manifold: 
>
>  
>
>  
>
> ;; 2017-07-10 -- we