I wasn't aware of the group-by function. Being new to clojure, I spent
hours coming up with my complex function. Seeing the one line solution
really blew me away. Thanks!!
Follow up questions on my problem (my code again first):
;data is a list of records (maps)
(def data '({:a1 2 :a2 34 :a3 76 :a4 87},
{:a1 3 :a2 30 :a3 38 :a4 39},
{:a1 2 :a2 67 :a3 32 :a4 38},
{:a1 4 :a2 47 :a3 73 :a4 38},
{:a1 3 :a2 84 :a3 86 :a4 63}))
;result is a map, key = value of :a1, val = all records for that value
of a1
;as pointed out, result can be obtained by (group-by :a1 data)
(def result
'{2 [{:a1 2 :a2 34 :a3 76 :a4 87}, {:a1 2 :a2 67 :a3 32 :a4 39}],
;grouped a1 = 2
3 [{:a1 3 :a2 30 :a3 38 :a4 39}, {:a1 3 :a2 84 :a3 86 :a4 63}],
;grouped a1 = 3
4 [{:a1 4 :a2 47 :a3 73 :a4 38}] ;grouped :a1 = 4
})
;groups the data by :a1 (see result above)
(def data-g (group-by :a1 data)
;for a key value in data-g returns the total of all :a4 values
(defn calc-tot-a4 [data-g k]
(reduce + (map :a4 (get data-g k))))
(defn adj-a4 [data k]
(assoc data k (map #(assoc % :a4 (+ (:a3 %) (:a4 %))) (get data k))))
(defn adj-data [data f]
(reduce f data (keys data)))
(def data-adj (adj-data data-g adj-a4))
I am expressing my domain specific problem in a very abstract way above.
In reality the data I am manipulating comes from a database. This is
financial performance data for an entire month. The data set is large -
about 50,000 records. I need to group it by "day" and then perform
manipulations on it. For example, I need to make adjustments to the
values loaded from the database based on certain conditions. I need to
aggregate values for a certain day etc.
My question is - am I on the right track with the adj-data/adj-a4
functions. Secondly since I am constantly modifying the in-memory data
(loaded using contrib.sql functions which loads it into plain maps), is
this inefficient since the data structures are immutable. So every time
I change a value clojure functions return me a new data structure. Will
this be inefficient when I am dealing with a large data set and making
constant modifications/adjustments to the values?
Thanks for your great help.
-----Original Message-----
From: clojure@googlegroups.com [mailto:clojure@googlegroups.com] On
Behalf Of Alan
Sent: Sunday, April 17, 2011 5:18 PM
To: Clojure
Subject: Re: Help with function to partition my data
Simpler still: group-by?
(group-by :a1 data) produces the result that Shoeb was looking for.
PS The original message isn't in Google's archives either.
On Apr 17, 7:54 am, Nate Austin <navis...@gmail.com> wrote:
> I didn't see this original email for some reason, but here's a
> simplified version:
>
> (reduce
> (fn [coll {a1 :a1 :as value}] (update-in coll [a1] conj value))
> {}
> data)
>
> This takes advantage of update-in passing nil if it doesn't have the
> key and conj with nil returning a seq of one value.
>
>
>
>
>
>
>
> On Sun, Apr 17, 2011 at 12:30 AM, Shoeb Bhinderwala
<shua...@gmail.com> wrote:
> > Wrote the following function which did the trick:
>
> > (defn partn [data]
> > (let
> > [add-mapping
> > (fn [m v]
> > (assoc m v (filter #(= v (:a1 %)) data)))]
> > (reduce add-mapping {} (set (map #(:a1 %) data))))
>
> > On Sat, Apr 16, 2011 at 8:15 PM, shuaybi <shua...@gmail.com> wrote:
>
> >> I am trying to write a function but can't get it right. I need to
loop
> >> through my data records and for each unique value of :a1 I need to
> >> partition the data and create a resulting map so that the key is
value
> >> of :a1 and the associated value of this key is all records with
that
> >> value of :a1. Better explained with code, where given the
definition
> >> of data below, I need to write a function that will return the
> >> definition of "result" below.
>
> >> ;data is a list of records (maps)
> >> (def data '({:a1 2 :a2 34 :a3 76 :a4 87},
> >> {:a1 3 :a2 30 :a3 38 :a4 39},
> >> {:a1 2 :a2 67 :a3 32 :a4 38},
> >> {:a1 4 :a2 47 :a3 73 :a4 38},
> >> {:a1 3 :a2 84 :a3 86 :a4 63}))
>
> >> ;result should be a map, keyed by unique values of :a1 and
containing
> >> all records for that value of :a1
> >> (def result '{2 ({:a1 2 :a2 34 :a3 76 :a4 87}, {:a1 2 :a2 67 :a3
> >> 32 :a4 39}), ;list of records where :a1 = 2
> >> 3 ({:a1 3 :a2 30 :a3 38 :a4 39}, {:a1 3 :a2 84 :a3
> >> 86 :a4 63}), ;list of records where :a1 = 3
> >> 4 ({:a1 4 :a2 47 :a3 73 :a4 38}) ;list of records
> >> where :a1 = 4
> >> })
>
> >> (defn get-val-a1
> >> "Return all records in data where :a1 is specified val"
> >> [data val]
> >> (filter
> >> #(= val (:a1 %)) ;filter function
> >> data))
>
> >> (defn unique-a1
> >> "Get set of all unique :a1 values"
> >> [m]
> >> (set (map #(:a1 %) m)))
>
> >> ;This is the function I am trying to write but can't get it right.
> >> (defn partn [data]
> >> (let [val-set (unique-a1 data)) ;get unique :a1 values
> >> ret-val {}]
> >> (loop [v val-set] ;loop through the unique values
> >> (assoc ret-val v (get-val-a1 data v))) ;build final map
> >> ret-val))
>
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Clojure" group.
> > To post to this group, send email to clojure@googlegroups.com
> > Note that posts from new members are moderated - please be patient
with your
> > first post.
> > To unsubscribe from this group, send email to
> > clojure+unsubscr...@googlegroups.com
> > For more options, visit this group at
> >http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with
your first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
