[ANN]: clj.jdbc 0.1-beta1 - Alternative implementation of jdbc wrapper for clojure.

[Andrey Antukh]

Hi!

I have some frustration with current official jdbc wrapper for clojure and
I have worked in one alternative mainly because of:

- Lack of documentation.
- Philosophical differences of how things should be done.

Documentation page: http://cljjdbc.readthedocs.org/en/latest/
Github: https://github.com/niwibe/clj.jdbc

Any feedback always welcome.

Andrey

-- 
Andrey Antukh - Андрей Антух - 
http://www.niwi.be/about.html
http://www.kaleidos.net/A5694F/

"Linux is for people who hate Windows, BSD is for people who love UNIX"
"Social Engineer -> Because there is no patch for human stupidity"

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Cornet 0.1.0 - Clojure asset pipeline for minification and compilation of LessCSS and CoffeeScript (...more coming soon)

[James Reeves]

Hi Marcin,

Why did you choose to group LESS compilations, CoffeeScript minification,
caching and resource minification in one middleware function? Why not have
a separate middleware function for each?

For example, instead of:

(compiled-assets-loader
 "precompiled"
 :lesscss-list ["css/bootstrap.less"]
 :mode :prod)


Consider instead a structure like:

(-> (route/resources "/*.less")
(wrap-compile-less)
(wrap-minify-css)
(wrap-cache (file-cache "tmp/less/compiled"))


You'd get the same functionality, but in a form where each function does a
specific task.

- James


On 16 November 2013 01:40, Marcin Skotniczny  wrote:

>
> 
> https://github.com/cosmi/cornet
>
> Cornet is a simple asset pipeline that will help you compile on the flight
> LessCSS, CoffeeScript and minify all your CSSes and JSes.
>
> I wasn't happy with what's already there 
> (dieter isn't
> really in spirit of Clojure) - I wanted something simple to use and
> compatible with Compojure. So here it is - my third iteration of the idea.
>
> There's simple basic usage described in the README, but I am really proud
> of the architecture of the library. It is very extendable and can combine
> transformations in any way you want, so I will work on new usage modes if
> needed (or you can figure it out by yourself :) ).
>
> Let me know what you think.
>
> Also, how are you managing this now?
>
>
> Marcin
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Can't get terminal parameters: Inappropriate ioctl for device error on clojure.java.shell

[Josh Kamau]

When i run this:
(clojure.java.shell/sh "chat" "-Vs" "' '" "'ATi'" "'OK'" "' '" ">"
"/dev/ttyUSB0" "<" "/dev/ttyUSB0")


I get this:
{:exit 2, :out "", :err "Can't get terminal parameters: Inappropriate ioctl
for device\n"}


When i run the command directly on the shell, it works. What is causing the
error? any options i could set?

Josh.

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Cornet 0.1.0 - Clojure asset pipeline for minification and compilation of LessCSS and CoffeeScript (...more coming soon)

[Marcin Skotniczny]

Well, I made sure you can do it:).  I created the grouped middleware 
functions for convenience's sake and I intend to add more.

Something like that will work:

(let [loader (resource-loader "/precompiled")]
(->
  (some-fn (wrap-lesscss-processor loader)
   (wrap-coffeescript-processor loader)
   (resource-loader "/public"))
  (cond-> devmode
(->
  (wrap-yui-css-compressor)
  (wrap-uglify-js-compressor)

Of course you will have to (use ...) more namespaces.

Although this might not be good enough, because:

   1. LessCSS files should be listed. You will usually want to compile only 
   one or two of them, the rest will be @imported. SASS has convention to mark 
   those files  with underscore _ at the beginning of filename, but LESS 
   doesn't have any solution like that, so...
   2. If you specify files to compile, they will be precompiled and any 
   compilation errors will be found immediately as the app won't start.
   3. I tried separate caching layer, but it doesn't work so good. It gets 
   especially complex with LessCSS - you have to watch imported files for 
   changes too. Also, I want to add transformation with non-java tools. 
   Integrating with those usually is based on filesystem anyway.

Marcin


W dniu sobota, 16 listopada 2013 13:08:18 UTC+1 użytkownik James Reeves 
napisał:
>
> Hi Marcin,
>
> Why did you choose to group LESS compilations, CoffeeScript minification, 
> caching and resource minification in one middleware function? Why not have 
> a separate middleware function for each?
>
> For example, instead of:
>
> (compiled-assets-loader
>  "precompiled"
>  :lesscss-list ["css/bootstrap.less"]
>  :mode :prod)
>
>
> Consider instead a structure like:
>
> (-> (route/resources "/*.less")
> (wrap-compile-less)
> (wrap-minify-css)
> (wrap-cache (file-cache "tmp/less/compiled"))
>
>
> You'd get the same functionality, but in a form where each function does a 
> specific task.
>
> - James
>
>
> On 16 November 2013 01:40, Marcin Skotniczny 
> > wrote:
>
>>
>> 
>> https://github.com/cosmi/cornet
>>
>> Cornet is a simple asset pipeline that will help you compile on the 
>> flight LessCSS, CoffeeScript and minify all your CSSes and JSes.
>>
>> I wasn't happy with what's already there 
>> (dieter isn't 
>> really in spirit of Clojure) - I wanted something simple to use and 
>> compatible with Compojure. So here it is - my third iteration of the idea.
>>
>> There's simple basic usage described in the README, but I am really proud 
>> of the architecture of the library. It is very extendable and can combine 
>> transformations in any way you want, so I will work on new usage modes if 
>> needed (or you can figure it out by yourself :) ).
>>
>> Let me know what you think.
>>
>> Also, how are you managing this now?
>>
>>
>> Marcin
>>
>> -- 
>> -- 
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clo...@googlegroups.com
>> Note that posts from new members are moderated - please be patient with 
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+u...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to clojure+u...@googlegroups.com .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN]: clj.jdbc 0.1-beta1 - Alternative implementation of jdbc wrapper for clojure.

[Michael Klishin]

2013/11/16 Andrey Antukh 

> - Lack of documentation.
> - Philosophical differences of how things should be done.
>
> Documentation page: http://cljjdbc.readthedocs.org/en/latest/
>

+ ridiculous Clojure CA that keeps non-North America/EU contributors away.

Good job, Andrey. It's important to have more than 1 option for every
problem
and clojure.java.jdbc could certainly use  some well documented,
contributor friendly competition.
-- 
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

What does Clojure get from Java 8?

[Alex Miller]

Clojure <= 1.5 was built with Java 1.5. Clojure 1.6 is built with Java 1.6. 
Neither uses anything special from Java 1.7 or 1.8. The changes are just making 
sure that Clojure will work with Java 1.8. 

Alex

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

ANN: data.fressian, read and write Fressian from Clojure

[Alex Miller]

The ci build is setup and the 0.2.0-SNAPSHOT release is available. The API docs 
are also now online.

Alex

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Releasing Caribou today: Open Source Clojure Web Ecosystem

[Ronen Cohen]

This look great! Is there a sample app anywhere?

On Wednesday, November 13, 2013 1:52:10 AM UTC+2, Ryan Spangler wrote:
>
> Hello Clojure,
>
> Excited to announce today the release of Caribou!  http://let-caribou.in/
>
> We have been building web sites and web applications with it for over two 
> years now and improving it every day.  Currently we have four people 
> working on it and another ten using it to build things, so it is getting a 
> lot of real world testing.
>
> It has been designed as a collection of independent libraries that could 
> each be useful on their own, but which come together as a meaningful whole.
>
> We have been spending the last couple months getting it ready for a full 
> open source release, and I am happy to say it is finally ready.  Funded and 
> supported by Instrument in Portland, OR:  http://weareinstrument.com/  We 
> have four projects using it in production, and several more about to be 
> launched (as well as over a dozen internal things).
>
> Documentation is here:  http://caribou.github.io/caribou/docs/outline.html
>
> Source is here:  http://github.com/caribou/caribou (use this for issues, 
> you don't actually need the source as it is installed through a lein 
> template).
>
> Some of the independently useful libraries Caribou is built on are:
>
> * Polaris -- Routing with data (not macros) and reverse routing! :  
> https://github.com/caribou/polaris
> * Lichen -- Image resizing to and from s3 or on disk: 
> https://github.com/caribou/lichen
> * Schmetterling -- Debugging Clojure processes from the browser:  
> https://github.com/prismofeverything/schmetterling
> * Antlers -- Useful extensions to mustache templating (helpers and blocks, 
> among other things):  https://github.com/caribou/antlers
> * Groundhog -- Replay http requests: 
> https://github.com/noisesmith/groundhog
>
> And many others.
>
> Basically this is an Alpha release, and I am announcing it here first in 
> order to get as much feedback from the community as possible.  We have made 
> it as useful as we can for our purposes and recognize that for it to 
> improve from here, we really need as many people using it and building 
> things with it as possible.  The documentation also needs to be put through 
> its paces:  we need to see how well people are able to use it who know 
> nothing about it, based only on the existing docs.
>
> All feedback welcome!  
>
> Thanks for reading!  I hope you find it useful.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN]: clj.jdbc 0.1-beta1 - Alternative implementation of jdbc wrapper for clojure.

[Brian Craft]

On Saturday, November 16, 2013 5:11:05 AM UTC-8, Michael Klishin wrote:
>
> + ridiculous Clojure CA that keeps non-North America/EU contributors away.
>
>
What is this? A link will do, if this has been discussed previously. 

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN]: clj.jdbc 0.1-beta1 - Alternative implementation of jdbc wrapper for clojure.

[Michael Klishin]

2013/11/16 Brian Craft 

> What is this? A link will do, if this has been discussed previously.


https://groups.google.com/forum/#!searchin/clojure/CA$20/clojure/FlwqULYM7n0/x1-ArtQe1isJ
https://groups.google.com/d/topic/clojure/0gwjKtatf-0/discussion

-- 
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] grid2d

[notofi]

Reviews are welcome:

https://github.com/damn/grid2d

This is the grid data structure that I use in my clojure RPG project: cyber 
dungeon quest . 

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN]: clj.jdbc 0.1-beta1 - Alternative implementation of jdbc wrapper for clojure.

[Michael Klishin]

2013/11/16 Andrey Antukh 

>
> - Lack of documentation.
>

FTR, there is some documentation for java.jdbc, but it certainly
isn't being actively worked on (despite not being covered by the CA) and may
already be out of date.

http://clojure-doc.org/articles/ecosystem/java_jdbc/home.html

If you'd like to contribute:
http://github.com/clojuredocs/guides
-- 
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Cornet 0.1.0 - Clojure asset pipeline for minification and compilation of LessCSS and CoffeeScript (...more coming soon)

[James Reeves]

On 16 November 2013 13:02, Marcin Skotniczny  wrote:

> Well, I made sure you can do it:).  I created the grouped middleware
> functions for convenience's sake and I intend to add more.
>

Ah, interesting! That's a design I'm more comfortable with.


> Although this might not be good enough, because:
>
>1. LessCSS files should be listed. You will usually want to compile
>only one or two of them, the rest will be @imported. SASS has convention to
>mark those files  with underscore _ at the beginning of filename, but LESS
>doesn't have any solution like that, so...
>2. If you specify files to compile, they will be precompiled and any
>compilation errors will be found immediately as the app won't start.
>
> Isn't this pretty easy to do by pushing through a request at
initialisation? It would have the added side benefit of pre-warming the
cache as well.


>
>1. I tried separate caching layer, but it doesn't work so good. It
>gets especially complex with LessCSS - you have to watch imported files for
>changes too. Also, I want to add transformation with non-java tools.
>Integrating with those usually is based on filesystem anyway.
>
>
Why doesn't it work well? Have you tried taking advantage of the 304
not-modified header?

For instance:

(-> less-route
wrap-not-modified
wrap-compile-less
(wrap-cache (file-cache "tmp"))


In this case, the "less-route" function returns a response with a
"Last-Modified" header in the response. If the request contains a
"If-Modified-Since" header, then the wrap-not-modified middleware can
replace the normal response with a 304 not-modified response.

Then, all we have to do is have the wrap-compile-less middleware pass up
any 304 responses without touching them, and the cache can then replace the
304 response with the cached version instead.

I'm working on a ring-cache library that will handle caching in a fairly
generic fashion by taking advantage of the infrastructure HTTP already has
for handling this. The core middleware looks like:

(defn wrap-cache [handler cache]
  (fn [request]
(if (not= (:request-method request) :get)
  (handler request)
  (let [timestamp (java.util.Date.)
cached(retrieve cache (:uri request))
request   (-> request (if-modified-since (:timestamp cached)))
response  (handler request)]
(if (not-modified? response)
  (:response cached)
  (do (store! cache (:uri request) {:response response :timestamp
timestamp})
  response))


- James

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN]: clj.jdbc 0.1-beta1 - Alternative implementation of jdbc wrapper for clojure.

[Andrey Antukh]

2013/11/16 Michael Klishin 

>
> 2013/11/16 Andrey Antukh 
>
>>
>> - Lack of documentation.
>>
>
> FTR, there is some documentation for java.jdbc, but it certainly
> isn't being actively worked on (despite not being covered by the CA) and
> may
> already be out of date.
>
> http://clojure-doc.org/articles/ecosystem/java_jdbc/home.html
>

Thanks! I know about existence of it! Is totally outdated with current
version of java.jdbc and incomplete ;)

Andrey

-- 
Andrey Antukh - Андрей Антух - 
http://www.niwi.be/about.html
http://www.kaleidos.net/A5694F/

"Linux is for people who hate Windows, BSD is for people who love UNIX"
"Social Engineer -> Because there is no patch for human stupidity"

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Cornet 0.1.0 - Clojure asset pipeline for minification and compilation of LessCSS and CoffeeScript (...more coming soon)

[Marcin Skotniczny]

Cornet works on lower layer and does not use ring-spec or HTTP. All the 
functions and middleware in Cornet take path as String and return 
java.net.URL (either "file:/..." or "jar:file:/") [as a side note: at some 
point I want it to be able to return data as InputStream or String instead 
of urls. ] For integrating with ring, there's wrap-url-response function 
that creates Compojure's (GET "/*"...) route out of Cornet function.

By caching I mean two work modes:
- in production mode I assume all the assets are truly static. They are 
compiled once, unless for some reason the cache is cleared (I had that 
problem once, as /tmp directory was swiped on production server in some 
cron job)
- in dev mode, the files will recompile every time you change the file 
(based on .lastModified value). For LESS I've been able to hack into 
compiler (thanks to Rhino engine) and Cornet will also note all the 
@imported files and watch them too. If the source is not a file (instead 
resides in a jar) then it is assumed a non-variable file.

Anyway, I didn't want to deal with ring responses, as ring and compojure 
are doing it well enough. Handling on ring response level was actually one 
of the things I didn't like with dieter.

- 
Marcin

W dniu sobota, 16 listopada 2013 18:30:03 UTC+1 użytkownik James Reeves 
napisał:
>
> On 16 November 2013 13:02, Marcin Skotniczny 
> > wrote:
>
>> Well, I made sure you can do it:).  I created the grouped middleware 
>> functions for convenience's sake and I intend to add more.
>>
>
> Ah, interesting! That's a design I'm more comfortable with.
>  
>
>> Although this might not be good enough, because:
>>
>>1. LessCSS files should be listed. You will usually want to compile 
>>only one or two of them, the rest will be @imported. SASS has convention 
>> to 
>>mark those files  with underscore _ at the beginning of filename, but 
>> LESS 
>>doesn't have any solution like that, so... 
>>2. If you specify files to compile, they will be precompiled and any 
>>compilation errors will be found immediately as the app won't start.
>>
>> Isn't this pretty easy to do by pushing through a request at 
> initialisation? It would have the added side benefit of pre-warming the 
> cache as well.
>  
>
>>
>>1. I tried separate caching layer, but it doesn't work so good. It 
>>gets especially complex with LessCSS - you have to watch imported files 
>> for 
>>changes too. Also, I want to add transformation with non-java tools. 
>>Integrating with those usually is based on filesystem anyway. 
>>
>>
> Why doesn't it work well? Have you tried taking advantage of the 304 
> not-modified header?
>
> For instance:
>
>  (-> less-route
> wrap-not-modified
> wrap-compile-less
> (wrap-cache (file-cache "tmp"))
>
>
> In this case, the "less-route" function returns a response with a 
> "Last-Modified" header in the response. If the request contains a 
> "If-Modified-Since" header, then the wrap-not-modified middleware can 
> replace the normal response with a 304 not-modified response.
>
> Then, all we have to do is have the wrap-compile-less middleware pass up 
> any 304 responses without touching them, and the cache can then replace the 
> 304 response with the cached version instead.
>  
> I'm working on a ring-cache library that will handle caching in a fairly 
> generic fashion by taking advantage of the infrastructure HTTP already has 
> for handling this. The core middleware looks like:
>
> (defn wrap-cache [handler cache]
>   (fn [request]
> (if (not= (:request-method request) :get)
>   (handler request)
>   (let [timestamp (java.util.Date.)
> cached(retrieve cache (:uri request))
> request   (-> request (if-modified-since (:timestamp cached)))
> response  (handler request)]
> (if (not-modified? response)
>   (:response cached)
>   (do (store! cache (:uri request) {:response response :timestamp 
> timestamp})
>   response))
>
>
> - James
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[ANN] overload-middleware 0.1.1

[Rob Day]

Hi all,

I've just published the first working version of a Ring middleware that 
some of you might find useful. It's designed for web apps where, if 
you're overloaded, it's better to serve some requests quickly and fail 
the others than to try and serve all the requests and do it slowly. (My 
background is in telecoms, where that's often the best approach.)


Specifically, you specify a target latency that you want 90% of requests 
to try and meet, and it applies the algorithm from 
http://www.eecs.harvard.edu/%7Emdw/papers/control-usits03.pdf to try and 
keep your latency below that threshold. The exact parameters of the 
algorithm are tunable, and different URLs or groups of URLs can have 
different targets and parameters.


It still needs a bit of tidying - docstrings, cleaning up some of the 
Midje tests, and so on - but it works and I think the documentation is 
usable, so I'm announcing it now.


Github (w/ docs): https://github.com/rkday/overload-middleware
Clojars: https://clojars.org/overload-middleware

Let me know if you have any feedback!
Rob

P.S. I threw together a simple demo (available from 
https://github.com/rkday/overload-middleware-demo) and tested it It 
works as designed - the URL wrapped in the overload middleware fails 
about 50% of requests, and so for the ones that succeed there's a 
noticeable speed improvement (80% of results are served in 120ms as 
opposed to 233ms for the unwrapped URL), though I suspect ab might be 
counting the immediate 503s in this result and throwing it off. If 
anyone has ideas for better tests, let me know!


2049 14:05:02 overload-demo $ for i in unwrapped wrapped; do ab -n 5 
-c 300 -q http://localhost:3000/$i; done

...
Document Path:  /unwrapped
..
Failed requests:0
...
Percentage of the requests served within a certain time (ms)
  50%178
  66%204
  75%216
  80%233
  90%358
  95%   1117
  98%   1211
  99%   3097
 100%  15196 (longest request)

...
Document Path:  /wrapped
...
Failed requests:25285
   (Connect: 0, Receive: 0, Length: 25285, Exceptions: 0)
Write errors:   0
Non-2xx responses:  25285
...
Percentage of the requests served within a certain time (ms)
  50% 67
  66% 86
  75%108
  80%120
  90%   1028
  95%   1080
  98%   1287
  99%   3082
 100%  15120 (longest request)

There's also no noticeable overhead in non-overload cases:

2050 14:07:30 overload-demo $ for i in unwrapped wrapped; do ab -n 50 -c 
1 -q http://localhost:3000/$i; done

...
Document Path:  /unwrapped
..
Failed requests:0
...
Percentage of the requests served within a certain time (ms)
  50% 33
  66% 33
  75% 33
  80% 33
  90% 33
  95% 34
  98% 35
  99% 35
 100% 35 (longest request)

...
Document Path:  /wrapped
...
Failed requests:0
...
Percentage of the requests served within a certain time (ms)
  50% 33
  66% 34
  75% 34
  80% 34
  90% 34
  95% 34
  98% 34
  99% 34
 100% 34 (longest request)

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Do web apps need Clojure?

[Brian Craft]

Marcus -- I hope you will post updates to the list with your experiences. 
It would be very interesting.

This thread has drifted a bit from (roughly) "What can you do with clojure 
web tooling?" toward "What can you imagine some day doing with the clojure 
web tooling of the future?", which are both interesting questions, but have 
somewhat different audiences. And the answers inform each other.
 

On Friday, November 15, 2013 8:20:32 AM UTC-8, Marcus Blankenship wrote:
>
> Me too!  Thanks to everyone who’s contributed, it’s been *very* helpful!
>
> On Nov 14, 2013, at 10:43 AM, Waldemar Schwan 
> > 
> wrote:
>
> I just want to say that this is one of the most interesting discussions I 
> followed on this mailing list.
>
> Thanks to all participants.
>
> Am 14.11.2013 um 19:24 schrieb Brian Craft 
> >:
>
>
>
> On Thursday, November 14, 2013 9:42:52 AM UTC-8, Jonathan Irving wrote:
>>
>> I agree with much of what you write James - I'm paid to write rails and 
>> node.js code, and I'm finding that node is encouraging me to compose 
>> small components and basically sidestep a lot of the issues that rails 
>> is designed to address. 
>>
>
> Can you give a concrete example? 
>
> -- 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>
> -- 
> -- 
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com 
> Note that posts from new members are moderated - please be patient with 
> your first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com 
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> --- 
> You received this message because you are subscribed to the Google Groups 
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to clojure+u...@googlegroups.com .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
> marcus blankenship
> \\\ Partner, Problem Solver, Linear Thinker
> \\\ 541.805.2736 \ @justzeros \ skype:marcuscreo
>  
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

core.logic merge substitutions map?

[Mark]

I stumbled across Timothy Baldridge's excellent video 
explaining how to incorporate 
data sources into core.logic.  It reinvigorated my interest in using 
core.logic to query SQL RDBMS.  I'm stumbling on a pretty simple thing, I 
think.  I've got a table that has three columns, a single primary key 
column and two value columns.  Using the pattern Tim outlines in his video, 
I've got a relation function that takes three parameters, one for each 
column and I'm trying to work through the case where the primary key is 
ground and the value columns are lvars.  This translates to a query of the 
form SELECT v1, v2 FROM t WHERE pkey=?.  Of course, this returns two values 
that must be unified.  That's where I'm stuck.

I know I want to return a substitution map, but I have two lvars to unify.  
How do I merge the two substitution maps?

Sample code:
(defn pkey-v1-v2-o [pkey v1 v2]
  (fn [a]
(let [pkey (walk a pkey)
  v1 (walk a v1)
  v2(walk a v2)]
  (condp = [(not (lvar? pkey))
(not (lvar? v1))
(not (lvar? v2))]
[true false false ] (to-stream 
  (let [r (first (query ["SELECT v1, v2 FROM T 
WHERE pkey=?" pkey]))]
(some-merge-fn (unify a v1 (:v1 r))
   (unify a v2 (:v2 r)

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Cornet 0.1.0 - Clojure asset pipeline for minification and compilation of LessCSS and CoffeeScript (...more coming soon)

[James Reeves]

On 16 November 2013 17:55, Marcin Skotniczny  wrote:

> Cornet works on lower layer and does not use ring-spec or HTTP. All the
> functions and middleware in Cornet take path as String and return
> java.net.URL (either "file:/..." or "jar:file:/") [as a side note: at some
> point I want it to be able to return data as InputStream or String instead
> of urls. ] For integrating with ring, there's wrap-url-response function
> that creates Compojure's (GET "/*"...) route out of Cornet function.
>

Doesn't this approach risk unnecessary repetition?

If someone writes some caching middleware, or middleware that optimises
Javascript, or whatever else, it's not compatible with the new abstraction
you've introduced.

It would be difficult for me to use Cornet in my own projects, for
instance, because I'd have to maintain two separate systems for caching
compiled resources.

- James

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Cornet 0.1.0 - Clojure asset pipeline for minification and compilation of LessCSS and CoffeeScript (...more coming soon)

[Marcin Skotniczny]

> Doesn't this approach risk unnecessary repetition?
>
> I am not sure if I catch your drift... What I wanted to have is a direct 
replacement for clojure.java.io/resource function - so if at some layer you 
come down to using this function, you can just replace it with cornet 
generated handler -> the function signature (accepted arguments and 
returned value) is the same. I could use ring-spec or something similar, 
but I think this is simpler, IMHO more elegant and (I think) a bit more 
flexible.

As for the dangers of repetition:
Basing on most typical real-life usage, the only need for recompilation is 
during development, so I guess performance here is not important. The cache 
in Cornet is dynamic (that is: tracks when source file changes) only in 
development mode - so if you have some assets that change _during_ app 
uptime on production server, then the basic Cornet stack is not suitable 
for your needs (although in this case I think that development mode will be 
fast enough - tracking changes is based on single .lastModified queries).
Otherwise, Cornet will return file urls, so you can check their timestamp 
and return 304, just like you would with normal files. The problem I had 
with caching on higher layer is that it is tricky to track the original 
file if you have many transformations pipelined - especially if some of 
them can combine multiple files. 


> If someone writes some caching middleware, or middleware that optimises 
> Javascript, or whatever else, it's not compatible with the new abstraction 
> you've introduced.
>
>
Well, you can just write middleware that is compatibile with it :). Anyway, 
it shouldn't be difficult to create an adapter to ring-spec or whatever 
format the middleware would use, right?

 

> It would be difficult for me to use Cornet in my own projects, for 
> instance, because I'd have to maintain two separate systems for caching 
> compiled resources.
>
>
If you are already using some asset pipelining method you are comfortable 
with, then I guess you don't need Cornet :).

-
Marcin

>
>  

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Cedric Greevey]

I can think of very few web apps where this would be a desirable approach.
A user getting a spurious error in response to a URL that they *know* is
valid is just going to hammer on the "reload" button until they get a
correct response from the server. So the server will end up even more
congested than if it just responded sluggishly but otherwise normally to
the request.


On Sat, Nov 16, 2013 at 1:21 PM, Rob Day  wrote:

>  Hi all,
>
> I've just published the first working version of a Ring middleware that
> some of you might find useful. It's designed for web apps where, if you're
> overloaded, it's better to serve some requests quickly and fail the others
> than to try and serve all the requests and do it slowly. (My background is
> in telecoms, where that's often the best approach.)
>
> Specifically, you specify a target latency that you want 90% of requests
> to try and meet, and it applies the algorithm from
> http://www.eecs.harvard.edu/%7Emdw/papers/control-usits03.pdf to try and
> keep your latency below that threshold. The exact parameters of the
> algorithm are tunable, and different URLs or groups of URLs can have
> different targets and parameters.
>
> It still needs a bit of tidying - docstrings, cleaning up some of the
> Midje tests, and so on - but it works and I think the documentation is
> usable, so I'm announcing it now.
>
> Github (w/ docs): https://github.com/rkday/overload-middleware
> Clojars: https://clojars.org/overload-middleware
>
> Let me know if you have any feedback!
> Rob
>
> P.S. I threw together a simple demo (available from
> https://github.com/rkday/overload-middleware-demo) and tested it It works
> as designed - the URL wrapped in the overload middleware fails about 50% of
> requests, and so for the ones that succeed there's a noticeable speed
> improvement (80% of results are served in 120ms as opposed to 233ms for the
> unwrapped URL), though I suspect ab might be counting the immediate 503s in
> this result and throwing it off. If anyone has ideas for better tests, let
> me know!
>
> 2049 14:05:02 overload-demo $ for i in unwrapped wrapped; do ab -n 5
> -c 300 -q http://localhost:3000/$i; done
> ...
> Document Path:  /unwrapped
> ..
> Failed requests:0
> ...
> Percentage of the requests served within a certain time (ms)
>   50%178
>   66%204
>   75%216
>   80%233
>   90%358
>   95%   1117
>   98%   1211
>   99%   3097
>  100%  15196 (longest request)
>
> ...
> Document Path:  /wrapped
> ...
> Failed requests:25285
>(Connect: 0, Receive: 0, Length: 25285, Exceptions: 0)
> Write errors:   0
> Non-2xx responses:  25285
> ...
> Percentage of the requests served within a certain time (ms)
>   50% 67
>   66% 86
>   75%108
>   80%120
>   90%   1028
>   95%   1080
>   98%   1287
>   99%   3082
>  100%  15120 (longest request)
>
> There's also no noticeable overhead in non-overload cases:
>
> 2050 14:07:30 overload-demo $ for i in unwrapped wrapped; do ab -n 50 -c 1
> -q http://localhost:3000/$i; done
> ...
> Document Path:  /unwrapped
> ..
> Failed requests:0
> ...
> Percentage of the requests served within a certain time (ms)
>   50% 33
>   66% 33
>   75% 33
>   80% 33
>   90% 33
>   95% 34
>   98% 35
>   99% 35
>  100% 35 (longest request)
>
> ...
> Document Path:  /wrapped
> ...
> Failed requests:0
> ...
> Percentage of the requests served within a certain time (ms)
>   50% 33
>   66% 34
>   75% 34
>   80% 34
>   90% 34
>   95% 34
>   98% 34
>   99% 34
>  100% 34 (longest request)
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email

Re: [ANN] Cornet 0.1.0 - Clojure asset pipeline for minification and compilation of LessCSS and CoffeeScript (...more coming soon)

[James Reeves]

On 16 November 2013 19:46, Marcin Skotniczny  wrote:

>
> Doesn't this approach risk unnecessary repetition?
>>
>> I am not sure if I catch your drift... What I wanted to have is a direct
> replacement for clojure.java.io/resource function - so if at some layer
> you come down to using this function, you can just replace it with cornet
> generated handler -> the function signature (accepted arguments and
> returned value) is the same. I could use ring-spec or something similar,
> but I think this is simpler, IMHO more elegant and (I think) a bit more
> flexible.
>

That does make a lot of sense, and I admit I'm in two minds about whether
this isn't a better solution than basing it on Ring's architecture.

I think the key (perhaps only) advantage of using a request/response map
has is that you can provide more caching information via headers. This
means you can loosely couple the caching architecture from the actual
generation of your resources.

(And by caching, I include the general case of storing the compiled CSS and
JS files.)

Without this, caching needs to be more tightly coupled, but perhaps this
isn't necessarily huge disadvantage. Incidentally, I was planning on
introducing the following basic caching protocol in a ring-cache library:

(defprotocol Cache
  (store! [cache key value])
  (retrieve [cache key]))


I'm not certain if this would be useful for you to support, or whether all
your caching of compiled assets is going to be purely on the filesystem.

Basing on most typical real-life usage, the only need for recompilation is
> during development, so I guess performance here is not important. The cache
> in Cornet is dynamic (that is: tracks when source file changes) only in
> development mode - so if you have some assets that change _during_ app
> uptime on production server, then the basic Cornet stack is not suitable
> for your needs (although in this case I think that development mode will be
> fast enough - tracking changes is based on single .lastModified queries).
>

What about a case where you want to store the production files on some
external CDN?

I guess that could be considered as part of the deployment process, done
separately through some additional script. But by keeping it all
"in-house", one could generate URLs that would work for both development
and production purposes.



> Otherwise, Cornet will return file urls, so you can check their timestamp
> and return 304, just like you would with normal files. The problem I had
> with caching on higher layer is that it is tricky to track the original
> file if you have many transformations pipelined - especially if some of
> them can combine multiple files.
>
>
>> If someone writes some caching middleware, or middleware that optimises
>> Javascript, or whatever else, it's not compatible with the new abstraction
>> you've introduced.
>>
>>
> Well, you can just write middleware that is compatibile with it :).
> Anyway, it shouldn't be difficult to create an adapter to ring-spec or
> whatever format the middleware would use, right?
>

That's certainly true, it just feels a little iffy :)

Overall, Cornet looks to be a big step forward. I suspect our only point of
contention is on the architecture around storing the compiled data, and
even then I'm uncertain whether you don't have the best approach after all.

- James

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Craig]

Very interesting. I have a similar requirement, but not in serving web 
requests. I haven't looked under the covers of your module, but wonder if 
it could be decoupled from web/ring?

Craig

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[James Reeves]

It may be useful for certain web services. If the server gets overloaded by
a temporary spike, the clients could pick a random sleep time before trying
again.

- James


On 16 November 2013 21:15, Cedric Greevey  wrote:

> I can think of very few web apps where this would be a desirable approach.
> A user getting a spurious error in response to a URL that they *know* is
> valid is just going to hammer on the "reload" button until they get a
> correct response from the server. So the server will end up even more
> congested than if it just responded sluggishly but otherwise normally to
> the request.
>
>
> On Sat, Nov 16, 2013 at 1:21 PM, Rob Day  wrote:
>
>>  Hi all,
>>
>> I've just published the first working version of a Ring middleware that
>> some of you might find useful. It's designed for web apps where, if you're
>> overloaded, it's better to serve some requests quickly and fail the others
>> than to try and serve all the requests and do it slowly. (My background is
>> in telecoms, where that's often the best approach.)
>>
>> Specifically, you specify a target latency that you want 90% of requests
>> to try and meet, and it applies the algorithm from
>> http://www.eecs.harvard.edu/%7Emdw/papers/control-usits03.pdf to try and
>> keep your latency below that threshold. The exact parameters of the
>> algorithm are tunable, and different URLs or groups of URLs can have
>> different targets and parameters.
>>
>> It still needs a bit of tidying - docstrings, cleaning up some of the
>> Midje tests, and so on - but it works and I think the documentation is
>> usable, so I'm announcing it now.
>>
>> Github (w/ docs): https://github.com/rkday/overload-middleware
>> Clojars: https://clojars.org/overload-middleware
>>
>> Let me know if you have any feedback!
>> Rob
>>
>> P.S. I threw together a simple demo (available from
>> https://github.com/rkday/overload-middleware-demo) and tested it It
>> works as designed - the URL wrapped in the overload middleware fails about
>> 50% of requests, and so for the ones that succeed there's a noticeable
>> speed improvement (80% of results are served in 120ms as opposed to 233ms
>> for the unwrapped URL), though I suspect ab might be counting the immediate
>> 503s in this result and throwing it off. If anyone has ideas for better
>> tests, let me know!
>>
>> 2049 14:05:02 overload-demo $ for i in unwrapped wrapped; do ab -n 5
>> -c 300 -q http://localhost:3000/$i; done
>> ...
>> Document Path:  /unwrapped
>> ..
>> Failed requests:0
>> ...
>> Percentage of the requests served within a certain time (ms)
>>   50%178
>>   66%204
>>   75%216
>>   80%233
>>   90%358
>>   95%   1117
>>   98%   1211
>>   99%   3097
>>  100%  15196 (longest request)
>>
>> ...
>> Document Path:  /wrapped
>> ...
>> Failed requests:25285
>>(Connect: 0, Receive: 0, Length: 25285, Exceptions: 0)
>> Write errors:   0
>> Non-2xx responses:  25285
>> ...
>> Percentage of the requests served within a certain time (ms)
>>   50% 67
>>   66% 86
>>   75%108
>>   80%120
>>   90%   1028
>>   95%   1080
>>   98%   1287
>>   99%   3082
>>  100%  15120 (longest request)
>>
>> There's also no noticeable overhead in non-overload cases:
>>
>> 2050 14:07:30 overload-demo $ for i in unwrapped wrapped; do ab -n 50 -c
>> 1 -q http://localhost:3000/$i; done
>> ...
>> Document Path:  /unwrapped
>> ..
>> Failed requests:0
>> ...
>> Percentage of the requests served within a certain time (ms)
>>   50% 33
>>   66% 33
>>   75% 33
>>   80% 33
>>   90% 33
>>   95% 34
>>   98% 35
>>   99% 35
>>  100% 35 (longest request)
>>
>> ...
>> Document Path:  /wrapped
>> ...
>> Failed requests:0
>> ...
>> Percentage of the requests served within a certain time (ms)
>>   50% 33
>>   66% 34
>>   75% 34
>>   80% 34
>>   90% 34
>>   95% 34
>>   98% 34
>>   99% 34
>>  100% 34 (longest request)
>>
>> --
>> --
>> You received this message because you are subscribed to the Google
>> Groups "Clojure" group.
>> To post to this group, send email to clojure@googlegroups.com
>> Note that posts from new members are moderated - please be patient with
>> your first post.
>> To unsubscribe from this group, send email to
>> clojure+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/clojure?hl=en
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "Clojure" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to clojure+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please b

Re: [ANN] overload-middleware 0.1.1

[Cedric Greevey]

Web browsers don't "pick a random sleep time before trying again", though;
they display a 500 error page and the user promptly clicks "reload" while
making an exasperated sigh.

If the client is something other than a web browser, then we're no longer
talking about the web, but some other thing, like SMTP or whatever. And
then we're outside the scope of my earlier remark, which specifically
mentioned the web. Not to mention outside the scope of the OP's product,
which was clearly referred to as a "Ring middleware" and not some other
sort of server component for some other kind of server.

Or were you meaning to suggest that the *users* pick a random sleep time
before trying again? Good luck herding them cats. See also
http://en.wikipedia.org/wiki/Wicked_problem ...


On Sat, Nov 16, 2013 at 4:24 PM, James Reeves  wrote:

> It may be useful for certain web services. If the server gets overloaded
> by a temporary spike, the clients could pick a random sleep time before
> trying again.
>
> - James
>
>
> On 16 November 2013 21:15, Cedric Greevey  wrote:
>
>> I can think of very few web apps where this would be a desirable
>> approach. A user getting a spurious error in response to a URL that they
>> *know* is valid is just going to hammer on the "reload" button until they
>> get a correct response from the server. So the server will end up even more
>> congested than if it just responded sluggishly but otherwise normally to
>> the request.
>>
>>
>> On Sat, Nov 16, 2013 at 1:21 PM, Rob Day  wrote:
>>
>>>  Hi all,
>>>
>>> I've just published the first working version of a Ring middleware that
>>> some of you might find useful. It's designed for web apps where, if you're
>>> overloaded, it's better to serve some requests quickly and fail the others
>>> than to try and serve all the requests and do it slowly. (My background is
>>> in telecoms, where that's often the best approach.)
>>>
>>> Specifically, you specify a target latency that you want 90% of requests
>>> to try and meet, and it applies the algorithm from
>>> http://www.eecs.harvard.edu/%7Emdw/papers/control-usits03.pdf to try
>>> and keep your latency below that threshold. The exact parameters of the
>>> algorithm are tunable, and different URLs or groups of URLs can have
>>> different targets and parameters.
>>>
>>> It still needs a bit of tidying - docstrings, cleaning up some of the
>>> Midje tests, and so on - but it works and I think the documentation is
>>> usable, so I'm announcing it now.
>>>
>>> Github (w/ docs): https://github.com/rkday/overload-middleware
>>> Clojars: https://clojars.org/overload-middleware
>>>
>>> Let me know if you have any feedback!
>>> Rob
>>>
>>> P.S. I threw together a simple demo (available from
>>> https://github.com/rkday/overload-middleware-demo) and tested it It
>>> works as designed - the URL wrapped in the overload middleware fails about
>>> 50% of requests, and so for the ones that succeed there's a noticeable
>>> speed improvement (80% of results are served in 120ms as opposed to 233ms
>>> for the unwrapped URL), though I suspect ab might be counting the immediate
>>> 503s in this result and throwing it off. If anyone has ideas for better
>>> tests, let me know!
>>>
>>> 2049 14:05:02 overload-demo $ for i in unwrapped wrapped; do ab -n 5
>>> -c 300 -q http://localhost:3000/$i; done
>>> ...
>>> Document Path:  /unwrapped
>>> ..
>>> Failed requests:0
>>> ...
>>> Percentage of the requests served within a certain time (ms)
>>>   50%178
>>>   66%204
>>>   75%216
>>>   80%233
>>>   90%358
>>>   95%   1117
>>>   98%   1211
>>>   99%   3097
>>>  100%  15196 (longest request)
>>>
>>> ...
>>> Document Path:  /wrapped
>>> ...
>>> Failed requests:25285
>>>(Connect: 0, Receive: 0, Length: 25285, Exceptions: 0)
>>> Write errors:   0
>>> Non-2xx responses:  25285
>>> ...
>>> Percentage of the requests served within a certain time (ms)
>>>   50% 67
>>>   66% 86
>>>   75%108
>>>   80%120
>>>   90%   1028
>>>   95%   1080
>>>   98%   1287
>>>   99%   3082
>>>  100%  15120 (longest request)
>>>
>>> There's also no noticeable overhead in non-overload cases:
>>>
>>> 2050 14:07:30 overload-demo $ for i in unwrapped wrapped; do ab -n 50 -c
>>> 1 -q http://localhost:3000/$i; done
>>> ...
>>> Document Path:  /unwrapped
>>> ..
>>> Failed requests:0
>>> ...
>>> Percentage of the requests served within a certain time (ms)
>>>   50% 33
>>>   66% 33
>>>   75% 33
>>>   80% 33
>>>   90% 33
>>>   95% 34
>>>   98% 35
>>>   99% 35
>>>  100% 35 (longest request)
>>>
>>> ...
>>> Document Path:  /wrapped
>>> ...
>>> Failed requests:0
>>> ...
>>> Percentage of the requests served within a certain time (ms)
>>>   50% 33
>>>   66% 34
>>>   75% 34
>>>   80% 34
>>>   90% 34
>>>   95% 34
>>>   98% 34
>>>   99% 34
>>>  100% 34 (longest request)
>>>
>>> --
>>> --
>>>

Re: [ANN] overload-middleware 0.1.1

[James Reeves]

On 16 November 2013 21:45, Cedric Greevey  wrote:

> Web browsers don't "pick a random sleep time before trying again", though;
> they display a 500 error page and the user promptly clicks "reload" while
> making an exasperated sigh.
>
> If the client is something other than a web browser, then we're no longer
> talking about the web, but some other thing, like SMTP or whatever. And
> then we're outside the scope of my earlier remark, which specifically
> mentioned the web. Not to mention outside the scope of the OP's product,
> which was clearly referred to as a "Ring middleware" and not some other
> sort of server component for some other kind of server.
>

It sounds like you've never heard of web services before.

Web servers are often used to serve information to clients other than web
browsers. HTTP is not a perfect protocol, but it's a well-supported one,
and is one of the most common ways for machines to communicate across the
internet. It's also used a lot in internal networks.

I can see overload-middleware being useful in an internal network, where
clients are submitting jobs to a central server. The clients can be
programmed to wait a random length of time before retrying if the server is
overloaded. I suspect that a better solution might be to use a queue, but
it really depends on the requirements of the system being developed.

- James

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Cedric Greevey]

On Sat, Nov 16, 2013 at 5:06 PM, James Reeves  wrote:

> Web servers are often used to serve information to clients other than web
> browsers.
>

[citation needed]

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Justin Smith]

I've use many services within apps that are json or xml results served over 
HTTP. This is a common way to integrate third party data or functionality 
into an app (especailly webapps or mobile apps).

On Saturday, November 16, 2013 2:49:46 PM UTC-8, Cedric Greevey wrote:
>
> On Sat, Nov 16, 2013 at 5:06 PM, James Reeves 
> 
> > wrote:
>
>> Web servers are often used to serve information to clients other than web 
>> browsers.
>>
>
> [citation needed] 
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Robert Day]

On 16/11/13 22:06, James Reeves wrote:


I can see overload-middleware being useful in an internal network, 
where clients are submitting jobs to a central server. The clients can 
be programmed to wait a random length of time before retrying if the 
server is overloaded. I suspect that a better solution might be to use 
a queue, but it really depends on the requirements of the system being 
developed.


Yes, this is the context in which I learnt about this approach to 
overload handling - one of the projects I work on is basically a simple 
HTTP interface to a more complicated protocol server (DIAMETER), with 
some database caching. It provides subscriber data to a VoIP server, so 
there's no direct user access to it - the "user interface" is a phone 
(or Skype-like program).


Queues are also a good solution in some cases, and we use them elsewhere 
- but you do need to tune your queue length to get the performance you 
want. The principle here is that you provide a target latency, and your 
"queue length" (actually the refill speed of your token bucket) gets 
automatically tuned to meet it.


(I probably caused some confusion initially by saying "web apps" - I 
should have said "HTTP services" which is what I actually had in mind.)


Rob

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[James Reeves]

On 16 November 2013 22:49, Cedric Greevey  wrote:

> On Sat, Nov 16, 2013 at 5:06 PM, James Reeves wrote:
>
>> Web servers are often used to serve information to clients other than web
>> browsers.
>>
>
> [citation needed]
>

Just google "web services". This is a *really* common use-case for HTTP.

- James

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Rob Day]

On 16/11/13 21:22, Craig wrote:


Very interesting. I have a similar requirement, but not in serving web 
requests. I haven't looked under the covers of your module, but wonder 
if it could be decoupled from web/ring?


Craig


Yes - I don't think that would be at all difficult. The only 
HTTP-specific bit is that I expect {:status 503} to mean "Overloaded" - 
assuming you already have functions that take a request and return a 
response, something that the function should return if it's overloaded 
(i.e. the analogue to {:status 503}) and (optionally) some way of 
telling if the wrapped function is reporting overload (the equivalent of 
#(= 503 (:status response)) then it should be quite easy to apply it to 
other protocols.


Rob

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] Cornet 0.1.0 - Clojure asset pipeline for minification and compilation of LessCSS and CoffeeScript (...more coming soon)

[Marcin Skotniczny]

> What about a case where you want to store the production files on some 
> external CDN?
>
> I guess that could be considered as part of the deployment process, done 
> separately through some additional script. But by keeping it all 
> "in-house", one could generate URLs that would work for both development 
> and production purposes.
>
>
My latest experience with high throughput sites is that you can just serve 
this kind of assets through some HTTP reverse proxy cache like Varnish 
Cache. Separate CDNs are starting to shine if you want to serve 
non-cacheable resources, like millions of photos etc.

Anyway, I was thinking earlier about basing it on ring-spec protocol, so 
you've got me to consider the advantages again... I want to keep it 
practical - the features implemented now do not require a very high level 
of abstraction, so I don't want to get caught in a trap of overthinking it 
:).

-
Marcin

>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Question on Sequences

[Alexandru Nedelcu]

Hi,

I'm trying to understand the design of Clojure's collections and one
thing that I find odd is the return of functions operating on sequences.
Like for instance a call such as this will return a lazy-seq and not a
vector:

(drop 2 [1 2 3 4])

The reason why I find it odd is that data-structures have different
characteristics and one may want to use a vector because it supports
efficient indexing and appending to the end.

Of course, dropping 2 elements like above from a vector is probably
going to have O(n) complexity and thus returning something lazy is more
appropriate. And while there are some operations, like "conj", "pop" and
"peek" that preserve the type, functions such as map and filter also
return lazy-seq. And I worry that the property of the collection you
start with is lost, given that this returns a "cons":

 (conj (filter even? [1 2 3 4 5]) 6)

So lets say that I want to write a generic function that preserves the
type of that collection. Is something like this idiomatic?

 (defn only-evens [coll]
   (into (empty coll) (filter even? coll)))

Thanks,

-- 
Alexandru Nedelcu
www.bionicspirit.com

PGP Public Key:
https://bionicspirit.com/key.aexpk



signature.asc
Description: OpenPGP digital signature

Re: [ANN] overload-middleware 0.1.1

[Colin Fleming]

>
> Web browsers don't "pick a random sleep time before trying again", though;
> they display a 500 error page and the user promptly clicks "reload" while
> making an exasperated sigh.
>

That sounds like "picking a random sleep time before trying again" to me :-)


>  If the client is something other than a web browser, then we're no longer
> talking about the web, but some other thing, like SMTP or whatever. And
> then we're outside the scope of my earlier remark, which specifically
> mentioned the web. Not to mention outside the scope of the OP's product,
> which was clearly referred to as a "Ring middleware" and not some other
> sort of server component for some other kind of server.
>
> Or were you meaning to suggest that the *users* pick a random sleep time
> before trying again? Good luck herding them cats. See also
> http://en.wikipedia.org/wiki/Wicked_problem ...
>
>
>
> On Sat, Nov 16, 2013 at 4:24 PM, James Reeves wrote:
>
>> It may be useful for certain web services. If the server gets overloaded
>> by a temporary spike, the clients could pick a random sleep time before
>> trying again.
>>
>> - James
>>
>>
>> On 16 November 2013 21:15, Cedric Greevey  wrote:
>>
>>> I can think of very few web apps where this would be a desirable
>>> approach. A user getting a spurious error in response to a URL that they
>>> *know* is valid is just going to hammer on the "reload" button until they
>>> get a correct response from the server. So the server will end up even more
>>> congested than if it just responded sluggishly but otherwise normally to
>>> the request.
>>>
>>>
>>> On Sat, Nov 16, 2013 at 1:21 PM, Rob Day  wrote:
>>>
  Hi all,

 I've just published the first working version of a Ring middleware that
 some of you might find useful. It's designed for web apps where, if you're
 overloaded, it's better to serve some requests quickly and fail the others
 than to try and serve all the requests and do it slowly. (My background is
 in telecoms, where that's often the best approach.)

 Specifically, you specify a target latency that you want 90% of
 requests to try and meet, and it applies the algorithm from
 http://www.eecs.harvard.edu/%7Emdw/papers/control-usits03.pdf to try
 and keep your latency below that threshold. The exact parameters of the
 algorithm are tunable, and different URLs or groups of URLs can have
 different targets and parameters.

 It still needs a bit of tidying - docstrings, cleaning up some of the
 Midje tests, and so on - but it works and I think the documentation is
 usable, so I'm announcing it now.

 Github (w/ docs): https://github.com/rkday/overload-middleware
 Clojars: https://clojars.org/overload-middleware

 Let me know if you have any feedback!
 Rob

 P.S. I threw together a simple demo (available from
 https://github.com/rkday/overload-middleware-demo) and tested it It
 works as designed - the URL wrapped in the overload middleware fails about
 50% of requests, and so for the ones that succeed there's a noticeable
 speed improvement (80% of results are served in 120ms as opposed to 233ms
 for the unwrapped URL), though I suspect ab might be counting the immediate
 503s in this result and throwing it off. If anyone has ideas for better
 tests, let me know!

 2049 14:05:02 overload-demo $ for i in unwrapped wrapped; do ab -n
 5 -c 300 -q http://localhost:3000/$i; done
 ...
 Document Path:  /unwrapped
 ..
 Failed requests:0
 ...
 Percentage of the requests served within a certain time (ms)
   50%178
   66%204
   75%216
   80%233
   90%358
   95%   1117
   98%   1211
   99%   3097
  100%  15196 (longest request)

 ...
 Document Path:  /wrapped
 ...
 Failed requests:25285
(Connect: 0, Receive: 0, Length: 25285, Exceptions: 0)
 Write errors:   0
 Non-2xx responses:  25285
 ...
 Percentage of the requests served within a certain time (ms)
   50% 67
   66% 86
   75%108
   80%120
   90%   1028
   95%   1080
   98%   1287
   99%   3082
  100%  15120 (longest request)

 There's also no noticeable overhead in non-overload cases:

 2050 14:07:30 overload-demo $ for i in unwrapped wrapped; do ab -n 50
 -c 1 -q http://localhost:3000/$i; done
 ...
 Document Path:  /unwrapped
 ..
 Failed requests:0
 ...
 Percentage of the requests served within a certain time (ms)
   50% 33
   66% 33
   75% 33
   80% 33
   90% 33
   95% 34
   98% 35
   99% 35
  100% 35 (longest request)

 ...
 Document Path:  /wrapped
 ...
 Failed requests:0
 ...
 Percentage of the requ

Re: Question on Sequences

[Andy Fingerhut]

I don't know if it is idiomatic, but it certainly looks like a good way to
achieve the desired effect.

If you chained together calls to several functions like your example
only-evens, it would not be lazy, and it would build up a separate instance
of collections of the original type at each step of the way.  Likely it
would be more efficient to delay the conversion back to the original
collection type until after the last sequence operation you wanted to
perform.

Andy


On Sat, Nov 16, 2013 at 4:01 PM, Alexandru Nedelcu wrote:

> Hi,
>
> I'm trying to understand the design of Clojure's collections and one
> thing that I find odd is the return of functions operating on sequences.
> Like for instance a call such as this will return a lazy-seq and not a
> vector:
>
> (drop 2 [1 2 3 4])
>
> The reason why I find it odd is that data-structures have different
> characteristics and one may want to use a vector because it supports
> efficient indexing and appending to the end.
>
> Of course, dropping 2 elements like above from a vector is probably
> going to have O(n) complexity and thus returning something lazy is more
> appropriate. And while there are some operations, like "conj", "pop" and
> "peek" that preserve the type, functions such as map and filter also
> return lazy-seq. And I worry that the property of the collection you
> start with is lost, given that this returns a "cons":
>
>  (conj (filter even? [1 2 3 4 5]) 6)
>
> So lets say that I want to write a generic function that preserves the
> type of that collection. Is something like this idiomatic?
>
>  (defn only-evens [coll]
>(into (empty coll) (filter even? coll)))
>
> Thanks,
>
> --
> Alexandru Nedelcu
> www.bionicspirit.com
>
> PGP Public Key:
> https://bionicspirit.com/key.aexpk
>
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: core.logic merge substitutions map?

[Mark]

d'oh!  Answering my own question:  Just compose the unify functions a la 
(unify (unify a v1 (:v1 r) v2 (:v2 r))

I have a feeling there is a library function/macro that would make this 
less messy but I can't find it from the cheatsheet.

On Saturday, November 16, 2013 10:31:50 AM UTC-8, Mark wrote:
>
> I stumbled across Timothy Baldridge's excellent video 
> explaining how to incorporate 
> data sources into core.logic.  It reinvigorated my interest in using 
> core.logic to query SQL RDBMS.  I'm stumbling on a pretty simple thing, I 
> think.  I've got a table that has three columns, a single primary key 
> column and two value columns.  Using the pattern Tim outlines in his video, 
> I've got a relation function that takes three parameters, one for each 
> column and I'm trying to work through the case where the primary key is 
> ground and the value columns are lvars.  This translates to a query of the 
> form SELECT v1, v2 FROM t WHERE pkey=?.  Of course, this returns two values 
> that must be unified.  That's where I'm stuck.
>
> I know I want to return a substitution map, but I have two lvars to 
> unify.  How do I merge the two substitution maps?
>
> Sample code:
> (defn pkey-v1-v2-o [pkey v1 v2]
>   (fn [a]
> (let [pkey (walk a pkey)
>   v1 (walk a v1)
>   v2(walk a v2)]
>   (condp = [(not (lvar? pkey))
> (not (lvar? v1))
> (not (lvar? v2))]
> [true false false ] (to-stream 
>   (let [r (first (query ["SELECT v1, v2 FROM T 
> WHERE pkey=?" pkey]))]
> (some-merge-fn (unify a v1 (:v1 r))
>(unify a v2 (:v2 r)
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Cedric Greevey]

The distribution will be narrow and peak at around 1 second, though, which
may not be what you want. Of course, the OP has since indicated that he
meant non-web uses of HTTP rather than serving web sites...



On Sat, Nov 16, 2013 at 7:10 PM, Colin Fleming
wrote:

> Web browsers don't "pick a random sleep time before trying again", though;
>> they display a 500 error page and the user promptly clicks "reload" while
>> making an exasperated sigh.
>>
>
> That sounds like "picking a random sleep time before trying again" to me
> :-)
>
>
>>  If the client is something other than a web browser, then we're no
>> longer talking about the web, but some other thing, like SMTP or whatever.
>> And then we're outside the scope of my earlier remark, which specifically
>> mentioned the web. Not to mention outside the scope of the OP's product,
>> which was clearly referred to as a "Ring middleware" and not some other
>> sort of server component for some other kind of server.
>>
>> Or were you meaning to suggest that the *users* pick a random sleep time
>> before trying again? Good luck herding them cats. See also
>> http://en.wikipedia.org/wiki/Wicked_problem ...
>>
>>
>>
>> On Sat, Nov 16, 2013 at 4:24 PM, James Reeves wrote:
>>
>>> It may be useful for certain web services. If the server gets overloaded
>>> by a temporary spike, the clients could pick a random sleep time before
>>> trying again.
>>>
>>> - James
>>>
>>>
>>> On 16 November 2013 21:15, Cedric Greevey  wrote:
>>>
 I can think of very few web apps where this would be a desirable
 approach. A user getting a spurious error in response to a URL that they
 *know* is valid is just going to hammer on the "reload" button until they
 get a correct response from the server. So the server will end up even more
 congested than if it just responded sluggishly but otherwise normally to
 the request.


 On Sat, Nov 16, 2013 at 1:21 PM, Rob Day  wrote:

>  Hi all,
>
> I've just published the first working version of a Ring middleware
> that some of you might find useful. It's designed for web apps where, if
> you're overloaded, it's better to serve some requests quickly and fail the
> others than to try and serve all the requests and do it slowly. (My
> background is in telecoms, where that's often the best approach.)
>
> Specifically, you specify a target latency that you want 90% of
> requests to try and meet, and it applies the algorithm from
> http://www.eecs.harvard.edu/%7Emdw/papers/control-usits03.pdf to try
> and keep your latency below that threshold. The exact parameters of the
> algorithm are tunable, and different URLs or groups of URLs can have
> different targets and parameters.
>
> It still needs a bit of tidying - docstrings, cleaning up some of the
> Midje tests, and so on - but it works and I think the documentation is
> usable, so I'm announcing it now.
>
> Github (w/ docs): https://github.com/rkday/overload-middleware
> Clojars: https://clojars.org/overload-middleware
>
> Let me know if you have any feedback!
> Rob
>
> P.S. I threw together a simple demo (available from
> https://github.com/rkday/overload-middleware-demo) and tested it It
> works as designed - the URL wrapped in the overload middleware fails about
> 50% of requests, and so for the ones that succeed there's a noticeable
> speed improvement (80% of results are served in 120ms as opposed to 233ms
> for the unwrapped URL), though I suspect ab might be counting the 
> immediate
> 503s in this result and throwing it off. If anyone has ideas for better
> tests, let me know!
>
> 2049 14:05:02 overload-demo $ for i in unwrapped wrapped; do ab -n
> 5 -c 300 -q http://localhost:3000/$i; done
> ...
> Document Path:  /unwrapped
> ..
> Failed requests:0
> ...
> Percentage of the requests served within a certain time (ms)
>   50%178
>   66%204
>   75%216
>   80%233
>   90%358
>   95%   1117
>   98%   1211
>   99%   3097
>  100%  15196 (longest request)
>
> ...
> Document Path:  /wrapped
> ...
> Failed requests:25285
>(Connect: 0, Receive: 0, Length: 25285, Exceptions: 0)
> Write errors:   0
> Non-2xx responses:  25285
> ...
> Percentage of the requests served within a certain time (ms)
>   50% 67
>   66% 86
>   75%108
>   80%120
>   90%   1028
>   95%   1080
>   98%   1287
>   99%   3082
>  100%  15120 (longest request)
>
> There's also no noticeable overhead in non-overload cases:
>
> 2050 14:07:30 overload-demo $ for i in unwrapped wrapped; do ab -n 50
> -c 1 -q http://localhost:3000/$i; done
> ...
> Document Path:  /unwrapped
> ..
> Failed requests:

Re: Question on Sequences

[Cedric Greevey]

This efficiency is why built-in functions to map, filter, etc. vectors to
vectors aren't provided; better to use (into []) on a chain of
seq-outputting transformations of a vector than to chain these hypothetical
vector-native functions.

OTOH, it occurs to me that efficient chaining can still be achieved in
library form, by using monads of some sort to transform it into (into []
(op1 (op2 (op3 ... v under the hood. We have something like (into []
(filter x v)), (into [] (map y v)), etc. that we'd like composed into (into
[] (filter x (map y (filter z ... v, rather than stacking repeated
(into [])s, and that pattern *looks* an awful lot like it should be
expressible as a monadic transformation of some kind. I don't have enough
monad-fu to write out an actual implementation, though. :)

That also leads to the weird thought that there might be some way to use
monads to create reducers-like functionality for nearly any underlying data
structure, with only the need to parametrize a few things for different
underlying storage.


On Sat, Nov 16, 2013 at 7:11 PM, Andy Fingerhut wrote:

> I don't know if it is idiomatic, but it certainly looks like a good way to
> achieve the desired effect.
>
> If you chained together calls to several functions like your example
> only-evens, it would not be lazy, and it would build up a separate instance
> of collections of the original type at each step of the way.  Likely it
> would be more efficient to delay the conversion back to the original
> collection type until after the last sequence operation you wanted to
> perform.
>
> Andy
>
>
> On Sat, Nov 16, 2013 at 4:01 PM, Alexandru Nedelcu 
> wrote:
>
>> Hi,
>>
>> I'm trying to understand the design of Clojure's collections and one
>> thing that I find odd is the return of functions operating on sequences.
>> Like for instance a call such as this will return a lazy-seq and not a
>> vector:
>>
>> (drop 2 [1 2 3 4])
>>
>> The reason why I find it odd is that data-structures have different
>> characteristics and one may want to use a vector because it supports
>> efficient indexing and appending to the end.
>>
>> Of course, dropping 2 elements like above from a vector is probably
>> going to have O(n) complexity and thus returning something lazy is more
>> appropriate. And while there are some operations, like "conj", "pop" and
>> "peek" that preserve the type, functions such as map and filter also
>> return lazy-seq. And I worry that the property of the collection you
>> start with is lost, given that this returns a "cons":
>>
>>  (conj (filter even? [1 2 3 4 5]) 6)
>>
>> So lets say that I want to write a generic function that preserves the
>> type of that collection. Is something like this idiomatic?
>>
>>  (defn only-evens [coll]
>>(into (empty coll) (filter even? coll)))
>>
>> Thanks,
>>
>> --
>> Alexandru Nedelcu
>> www.bionicspirit.com
>>
>> PGP Public Key:
>> https://bionicspirit.com/key.aexpk
>>
>>
>  --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[James Reeves]

On 17 November 2013 01:52, Cedric Greevey  wrote:

> The distribution will be narrow and peak at around 1 second, though, which
> may not be what you want. Of course, the OP has since indicated that he
> meant non-web uses of HTTP rather than serving web sites...
>

Web services are generally considered to be part of the web, hence the term
"*web* service" :)

- James

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN]: clj.jdbc 0.1-beta1 - Alternative implementation of jdbc wrapper for clojure.

[Zach Oakes]

Andrey, this looks interesting. It seems to be based on a really old 
version of clojure.java.jdbc, though. Is there a reason for this? Since it 
is a fork, it seems like it would be best to base it on a recent version, 
to aid those who may want to try switching to it. Also, does it require JDK 
7 to compile?

On Saturday, November 16, 2013 6:48:03 AM UTC-5, Andrey Antukh wrote:
>
> Hi!
>
> I have some frustration with current official jdbc wrapper for clojure and 
> I have worked in one alternative mainly because of:
>
> - Lack of documentation.
> - Philosophical differences of how things should be done.
>
> Documentation page: http://cljjdbc.readthedocs.org/en/latest/
> Github: https://github.com/niwibe/clj.jdbc
>
> Any feedback always welcome.
>
> Andrey
>
> -- 
> Andrey Antukh - Андрей Антух - >
> http://www.niwi.be/about.html
> http://www.kaleidos.net/A5694F/
>
> "Linux is for people who hate Windows, BSD is for people who love UNIX"
> "Social Engineer -> Because there is no patch for human stupidity" 
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Dave Ray]

FWIW, Netflix uses a sorta similar approach but the overload detection
lives on the client-side since different clients may have varying
definitions of "slow", may want finer grained control of fallback behavior,
etc:

  http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html

There's a library for this:

  https://github.com/Netflix/Hystrix

and Clojure bindings:

  https://github.com/Netflix/Hystrix/tree/master/hystrix-contrib/hystrix-clj

Cheers,

Dave



On Sat, Nov 16, 2013 at 6:35 PM, James Reeves  wrote:

> On 17 November 2013 01:52, Cedric Greevey  wrote:
>
>> The distribution will be narrow and peak at around 1 second, though,
>> which may not be what you want. Of course, the OP has since indicated that
>> he meant non-web uses of HTTP rather than serving web sites...
>>
>
> Web services are generally considered to be part of the web, hence the
> term "*web* service" :)
>
> - James
>
> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clojure@googlegroups.com
> Note that posts from new members are moderated - please be patient with
> your first post.
> To unsubscribe from this group, send email to
> clojure+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [ANN] overload-middleware 0.1.1

[Cedric Greevey]

On Sat, Nov 16, 2013 at 9:35 PM, James Reeves  wrote:

> On 17 November 2013 01:52, Cedric Greevey  wrote:
>
>> The distribution will be narrow and peak at around 1 second, though,
>> which may not be what you want. Of course, the OP has since indicated that
>> he meant non-web uses of HTTP rather than serving web sites...
>>
>
> Web services are generally considered to be part of the web, hence the
> term "*web* service" :)
>

Well, which is it? Either it's the web, and the user will probably promptly
hit reload if faced with a 503 error at what should be a working URL, or
else it's not the web, and lies outside the scope of my original remark.

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: Do web apps need Clojure?

[Jarrod Swart]

I think the more interesting question is: *do web developers need Clojure? 
 *If performance were the sole concern then everyone would still code web 
apps in C or assembler.  There is great power in abstraction.  I like this 
talk by Uncle Bob: 
http://skillsmatter.com/podcast/agile-testing/bobs-last-language  If you 
don't have time to watch it I will summarize: every time we *get a more 
powerful programming environment* (for developers) we *reduce* our 
programming options (in the language).  

   - C --> Java == no pointers
   - Java --> Clojure == no immutability, etc.
   
Every major advancement for developers has largely involved reducing our 
options and choices.  Take away register manipulation, take away pointers, 
take away variables, and so on.  I highly recommend the talk by Uncle Bob.

So I highly encourage you to investigate the benefits that Clojure provide 
to the developer, over the benefits provided to the machine.

Good luck in your search!

Jarrod

On Wednesday, November 13, 2013 5:38:49 PM UTC-5, Marcus Blankenship wrote:
>
> Hi Folks,
>
> We’re a Python / Django shop, and some folks are getting excited about 
> using Clojure for building web apps.  Certainly there are numerous 
> open-source options to assist us (Pedastal, Ring, Compojure, Caribou, etc), 
> but I think it begs a larger question: *as a rule, do web applications 
> need the power that Clojure brings to the table?*
>
> Other folks on my team are telling me that solutions built with Python / 
> Django (or even RubyOnRails) fully satisfy the needs of 99% of the web apps 
> we have built, and that Clojure offers nothing new to this problem space.  
>
> So, here’s the question: *How are you are actually using Clojure, and why 
> did you choose to use it, particularly in the “web application” space? * 
>
> Thanks,
> Marcus
>
>
>
> marcus blankenship
> \\\ Partner, Problem Solver, Linear Thinker
> \\\ 541.805.2736 \ @justzeros \ skype:marcuscreo
>  
>

-- 
-- 
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your 
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--- 
You received this message because you are subscribed to the Google Groups 
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.