tps://github.com/damballa/abracad
[2] https://github.com/damballa/parkour
--
Marshall Bockrath-Vandegrift
Principal Software Engineer, Damballa R&D
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to cl
l or on the
offending other direct dependencies.
HTH,
--
Marshall Bockrath-Vandegrift
Principal Software Engineer, Damballa R&D
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
rnatively to the 1.7.0 changes mentioned by Alex
Miller, check out:
https://github.com/llasram/letterpress
If you can stomach the (very gentle) monkey-patching, it provides the
ability to arbitrarily redefine and hook printing within a particular
dynamic scope.
--
Marshall Bockrath-Vandegrift
Princ
a/parkour/blob/master/src/java/parkour/hadoop/Mappers.java
https://github.com/damballa/abracad/blob/master/src/java/abracad/avro/ClojureDatumReader.java
HTH,
--
Marshall Bockrath-Vandegrift
Principal Software Engineer, Damballa R&D
--
You received this message because you are subscribed to
On Tue, May 26, 2015 at 11:29 PM Alex Miller wrote:
> The point I was getting at is really whether you should consider this to
> be broken with the old behavior too.
>
Such APIs are tricky to use correctly from Clojure via seqs, but it is
possible to do so with the "normal" automatic clojure.la
ed via
the deep guts of a complex Java API suddenly became nonsensical.
On Tue, May 26, 2015 at 9:45 PM Marshall Bockrath-Vandegrift <
llas...@gmail.com> wrote:
> Ugh -- looks like the iterator value re-use behavior for EnumMap entrySet
> was "fixed" post Java 1.6 (my exa
n still make one if you want via (clojure.lang.IteratorSeq/create iter)
> but I would consider it deprecated. I think a custom lazy-seq or a
> loop-recur would be a better way to handle this case, which in my opinion
> is highly unusual. That said, my ears are open if this is an issue for a
&g
g else, or having expectations about its
> rate of consumption was as dubious before as it is now.
>
> Unless of course I misunderstand your intent, which possible because I am
> on a phone without easy access to look further at the commit and am going
> by memory.
>
>
>
Some of my code is broken by
commit c47e1bbcfa227723df28d1c9e0a6df2bcb0fecc1, which landed in
1.7.0-alpha6 (I lasted tested with -alpha5 and have been unfortunately busy
since). The culprit is the switch to producing seqs over iterators as
chunked iterators. This would appear to break seq-based t
https://github.com/damballa/parkour
--
Marshall Bockrath-Vandegrift
Principal Software Engineer, Damballa R&D
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that po
On Friday, January 30, 2015 at 5:00:31 PM UTC-5, Jason Wolfe wrote:
Thanks for the recommendation. For now we're looking for a simple
> low-level interface to MR, but we're also keeping an eye on parkour and
> pigpen for more complex tasks down the road. Can you explain why I might
> prefer p
Not a solution to your immediate problem, but if this is for
new development (not an existing mass of clojure-hadoop code), I'd suggest
looking at Parkour instead. As the main Parkour developer I'm obviously
biased, but Parkour exists in part because the compilation model used by
clojure-hadoo
At the very least, all the places where you switched to method/property
syntax now require runtime reflection. Try leaving those as they were, or
switching to using the keyword as the function (which is somewhat more
idiomatic). You could instead hint the appropriate object type in each
functi
On Monday, January 26, 2015 at 9:24:28 PM UTC-5, Dan Harbin wrote:
>
>
> io/file
>
>
Just delete that line. The `io/resource` function returns a URL which all
the Clojure IO functions can handle just fine-as is. When running in
development the URL happens to be a `file://` URL, and thus s
Clojure's tagged literals and `clojure.edn` namespace functions allow
hyper-local (per call) specification of tagged-literal data readers during
deserialization. The letterpress library provides a similar capability for
printing, allowing per-call specification of print methods and general
rec
What's your `print-dup` for instants print them as? The way compilation
for these expressions is going to work is:
(a) The initial form will be read using the configured *data-readers*,
handing the compiler a form with a literal instance object.
(b) The compiler will generate code to create tha
Hi all,
I’m pleased to announce the release of version 0.6.0 of Parkour. Parkour
is a Clojure Hadoop integration library, focused on support for writing
Hadoop MapReduce applications in plain Clojure. Code, documentation, and
artifact dependency information is available on Github:
https://gi
Parkour is a Clojure library for writing distributed programs in the
MapReduce pattern which run on the Hadoop MapReduce platform:
https://github.com/damballa/parkour
Release 0.5.4 adds significant new features for REPL integration.
Parkour now supports connecting to a live cluster, then runn
phillip.l...@newcastle.ac.uk (Phillip Lord) writes:
> I did consider the possibility that it just wasn't funny!
Oh, no – it was hilarious. :-)
-Marshall
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojur
our exclusively uses
reducers instead of lazy sequences, but I’m hoping shortly to add the
necessary glue to allow tasks to work via seqs too when desired.
[2] https://github.com/brandonbloom/transduce
--
Marshall Bockrath-Vandegrift
Principal Software Engineer, Damballa R&D
--
--
You re
over AWS EMR?
I have not run it live on EMR, but the unit test matrix includes Hadoop
versions 0.20.205, 1.0.3, and 2.2.0, which are the sufficiently-recent
Hadoop releases EMR’s documentation claims are supported.
--
Marshall Bockrath-Vandegrift
Principal Software Engineer, Damballa R&D
d will be more than happy to discuss Parkour
in detail with those interested.
Questions and pull requests welcome!
--
Marshall Bockrath-Vandegrift
Principal Software Engineer, Damballa R&D
--
--
You received this message because you are subscribed to the Google
Groups "Cloj
John Mastro writes:
> This isn't a very deep question, but I wonder every time I come across
> it: to what does "-dup" in `print-dup` and `*print-dup*` refer?
I don’t have any special knowledge in this regard, but I’ve always
thought of it as “duplicate,” which makes some sense when you think of
Mark writes:
> I think my preferred solution would be to allow imported Java classes
> to be aliased, so I could do this:
>
>> (import '(org.fooinstitute.team.library.foo package :as foop))
> => org.fooinstitute.team.library.foo.package
>> (foop/isFoo "foop")
> => false
>
> But to the best of my
Brian Craft writes:
> What's going on? Is there some other way to type hint the case with
> the threading macro?
I’m pretty sure this is CLJ-865 “Macroexpansion discards &form metadata”:
http://dev.clojure.org/jira/browse/CLJ-865
In which case you’ll need to use an alternative form of the c
Gary Zhao writes:
> I'm using core.async, but always see the following warnings. How do I
> suppress them?
It looks like you are literally `:use`ing `core.async`, or doing a
`:require ... :refer :all`. This is generally not what you want,
because you have no control over the vars now referred
Cedric Greevey writes:
> Erm ... a "MOOC"?
Massive Open Online Course – e.g., Coursera, edX, Udacity, etc. This is
for the Coursera-offered version of the University of Minnesota course
on recommendation systems, but that didn’t seem particularly relevant to
the problem at hand.
-Marshall
--
Hi all:
I was wondering – does anyone have any experience with or patterns for
integrating Clojure with existing Java dependency injection frameworks?
I’m working with the LensKit framework [1] for a MOOC and it uses a
JSR-330/javax.inject dependency-injection framework named `grapht` [2]
for, wel
h namespace to have separate nil-heap `reify`-cations of the
`Heap` protocol. All local invocations of heap protocol functions would
then go through proxy functions which would replace `nil` with the local
nil-heap implementation.
--
Marshall Bockrath-Vandegrift
Damballa Staff Software Engin
Vincent Chen writes:
> - Use something else than records to model structs (suggestions welcome)?
Maps.
Records have concrete Java types, which allows them to implement
interfaces and participate in protocols. Fields defined on a record
type are backed by JVM object fields, which can increase p
Philipp Meier writes:
> (alter-var-root #'*read-eval* (constantly false))
> => why do you think this is necessary?
Some versions of the Leiningen `app` template put this in the skeleton
initial source file. I assume that’s where this came from.
-Marshall
--
--
You received this message beca
Dennis Haupt writes:
> (defn fib-n [n]
> (let [fib (fn [a b] (cons a (lazy-seq (fib b (+ b a)]
> (take n (fib 1 1
>
> can't i do a recursion here? how can i achieve this without doing an
> "outer defn"?
You just need to give the anonymous function a name it can use to refer
to itself f
Clojars.
[1] http://avro.apache.org/
--
Marshall Bockrath-Vandegrift
Damballa Staff Software Engineer
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new m
drclj writes:
> Thanks everyone, in the apply function source code I see
>
> ([^clojure.lang.IFn f args]
> (. f (applyTo (seq args
>
> Seems the (applyTo (seq args)) returns arg parameters,
>
> And the f is invoked only once:
>
> (. f args)
I think you’re missing that `.` is a special form w
Shantanu Kumar writes:
> Wow! This is neat. Congratulations on the release.
Thank you!
> A minor observation: It may help some readers if you mention on the
> README that it may not work with lein-try (as I found) and that the
> user must `require` the ns first: (require '[method.fn])
Good poi
Do Clojure’s built-in methods for bridging the distinction between
functions and host platform methods seem clunky to you? Then the
method-fn library may hold the solution!:
(require 'method.fn)
(map #mf/i String/trim [" a" "b "]) ;; Look ma, no reflection!
(map #mf/s Math/log (ra
Jürgen Hötzel writes:
> My Question was about the interning. AFAIK interning shoult only
> return a "new" Symbol, when the Symbol wasn't interned already.
We may already be on the same page. I was just pointing out that there
aren’t any semantic benefits to that form of iterning for Symbols. T
Jürgen Hötzel writes:
> If a symbol X is interned twice, shouldn't the second Symbol.intern(X)
> return the previous interned symbol object?
Symbols in Clojure can have metadata, and so can’t have pure value-based
identity. Keywords fill that role instead, which is why keywords can’t
hold metad
the positions or working at Damballa,
please feel free to contact me directly at .
--
Marshall Bockrath-Vandegrift
Damballa Staff Software Engineer
--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to cl
"Jim - FooBar();" writes:
> CompilerException java.lang.UnsupportedOperationException: nth not
> supported on this type: Character, compiling:(NO_SOURCE_PATH:1:1)
If you examine the implementation of `extend-protocol` and for how it
distinguishes between additional functions being defined for a
Plínio Balduino writes:
> I'm writing a talk about Clojure in the real world and I would like to
> know, if possible, which companies are using Clojure for production or
> to make internal tools.
At Damballa we’re using Clojure and Cascalog to do all of our
Hadoop-based backend data processing.
"Jim - FooBar();" writes:
> Now, the first time I (load-file "xxx.core.clj") everything is
> perfectly fine. The minute I make a change and re-load I get:
>
> NoClassDefFoundError Could not initialize class yyy.Foo
This confuses me, because the JVM should only be loading and
initializing the Jav
kuba roth writes:
> I've got more examples for OSX which clearly shows that as soon as the
> number of tasks exceeds number of cores pmap performance suffers. It
> seems to me like there is no blocking taking place on threads and all
> the tasks are started at the same time.
There’s a still-open
Hi all:
As I myself missed when this group got started, I’d like to
delayed-announce the existence of atl-clj, a meetup group for Clojure
users in the Atlanta, GA area:
http://www.meetup.com/Atl-Clj/
Current plan is to meet ~monthly on second Tuesdays. The next scheduled
meetup is for Febru
larry google groups writes:
> Any suggestion, no matter how far fetched, will be welcome. I am
> ignorant about the JVM so I am having trouble debugging this problem.
> java -jar kiosk.clj 3
These are weird (.clj vs .jar), but since you say whatever you actually
ran worked...
Chas Emerick writes:
> Keeping the discussion here would make sense, esp. in light of
> meetup.com's horrible "discussion board".
Excellent. Solves the problem of deciding the etiquette of jumping on
the meetup board for a meetup one has never been involved in. :-)
> The nature of the `burn`
"Wm. Josiah Erikson" writes:
> Am I reading this right that this is actually a Java problem, and not
> clojure-specific? Wouldn't the rest of the Java community have noticed
> this? Or maybe massive parallelism in this particular way isn't
> something commonly done with Java in the industry?
>
>
Lee Spector writes:
> FWIW I used records for push-states at one point but did not observe a
> speedup and it required much messier code, so I reverted to
> struct-maps. But maybe I wasn't doing the right timings. I'm curious
> about how you changed to records without the messiness. I'll include
"Wm. Josiah Erikson" writes:
> I hope this helps people get to the bottom of things.
Not to the bottom of things yet, but found some low-hanging fruit –
switching the `push-state` from a struct-map to a record gives a flat
~2x speedup in all configurations I tested. So, that’s good?
I have how
kristianlm writes:
> I'm enjoying testing Java code with Clojure and it's been a lot of fun
> so far. Coming from Scheme, the transit is comfortable. However, I
> encountered a big surprise when I nested def's and used them with a
> proxy:
This is a common surprise for people with previous expos
cameron writes:
> the megamorphic call site hypothesis does sound plausible but I'm
> not sure where the following test fits in.
...
> I was toying with the idea of replacing the EmptyList class with a
> PersistsentList instance to mitigate the problem
> in at least one common case, however i
Andy Fingerhut writes:
> I'm not practiced in recognizing megamorphic call sites, so I could be
> missing some in the example code below, modified from Lee's original
> code. It doesn't use reverse or conj, and as far as I can tell
> doesn't use PersistentList, either, only Cons.
...
> Can you
Lee Spector writes:
> If the application does lots of "list processing" but does so with a
> mix of Clojure list and sequence manipulation functions, then one
> would have to write private, list/cons-only versions of all of these
> things? That is -- overstating it a bit, to be sure, but perhaps
Lee Spector writes:
> Is the following a fair characterization pending further developments?
>
> If you have a cons-intensive task then even if it can be divided into
> completely independent, long-running subtasks, there is currently no
> known way to get significant speedups by running the subt
"nicolas.o...@gmail.com" writes:
> What happens if your run it a third time at the end? (The question
> is related to the fact that there appears to be transition states
> between monomorphic and megamorphic call sites, which might lead to
> an explanation.)
Same results, but your comment jog
"Wm. Josiah Erikson" writes:
> Aha. Not only do I get a lot of "made not entrant", I get a lot of
> "made zombie". However, I get this for both runs with map and with
> pmap (and with pmapall as well)
I’m not sure this is all that enlightening. From what I can gather,
“made not entrant” just me
cameron writes:
> There does seem to be something unusual about conj and
> clojure.lang.PersistentList in this parallel test case and I don't
> think it's related to the JVMs memory allocation.
I’ve got a few more data-points, but still no handle on what exactly is
going on.
My last benchmark s
Andy Fingerhut writes:
> My current best guess is the JVM's memory allocator, not Clojure code.
I didn’t mean to imply the problem was in Clojure itself, but I don’t
believe the issue is in the memory allocator either. I now believe the
problem is in a class of JIT optimization HotSpot is perfo
cameron writes:
> Interesting problem, the slowdown seems to being caused by the reverse
> call (actually the calls to conj with a list argument).
Excellent analysis, sir! I think this points things in the right
direction.
> fast-reverse : map-ms: 3.3, pmap-ms 0.7, speedup 4.97
> list-cons
Lee Spector writes:
> I'm also aware that the test that produced the data I give below,
> insofar as it uses pmap to do the distribution, may leave cores idle
> for a bit if some tasks take a lot longer than others, because of the
> way that pmap allocates cores to threads.
Although it doesn’t i
60 matches
Mail list logo
