Re: Optimizing cross-product mappings

2009-06-05 Thread Sean Devlin
Sounds like a candidate for the daily WTF... On Jun 5, 12:54 pm, Daniel Jomphe wrote: > You guessed mostly right, Daniel :) This guy hashed some fields of his > client's database, replacing the original content with its hashed > version. I don't know everything, but he at least obfuscated the >

Re: Optimizing cross-product mappings

2009-06-05 Thread Daniel Jomphe
You guessed mostly right, Daniel :) This guy hashed some fields of his client's database, replacing the original content with its hashed version. I don't know everything, but he at least obfuscated the addresses. He (and the client who asked to "encrypt with MD5") thought he was actually encryptin

Re: Optimizing cross-product mappings

2009-06-05 Thread Daniel Lyons
Daniel, I guess I have to ask... why do you need to do this in the first place? :) Zip codes are already unique. Are you searching through data where someone else MD5'd some zip codes to obfuscate them? Just curious. — Daniel Lyons http://www.storytotell.org -- Tell It! --~--~-~--~

Re: Optimizing cross-product mappings

2009-06-05 Thread Daniel Jomphe
Stuart, I took a second look at your suggestion. I integrated it directly in my code, which became much more elegant as a direct consequence. :) And it's a tiny, tiny little bit faster too. :) (defn all-zips [] "Returns a lazy list of all possible American zipcodes." (let [zips5 (zip-format

Re: Optimizing cross-product mappings

2009-06-05 Thread Daniel Jomphe
Thanks for your replies. Sean: Type-hinting the String didn't help. And for some reason, defn'd function is 1.5x slower than my anonymous one. Daniel: Thanks for the info. Once my algorithms are fine (if they can get finer), I'll definitely narrow down the zips5 seed, and use your suggested MD5 d

Re: Optimizing cross-product mappings

2009-06-05 Thread Sean Devlin
On Jun 5, 11:14 am, Daniel Lyons wrote: > On Jun 5, 2009, at 8:56 AM, Daniel Jomphe wrote: > > > I need to generate a list of all possible American zipcodes, MD5- > > digested. Later on, I will need to do much more involving stuff, > > processor-wize, with this. But already, generating a naive

Re: Optimizing cross-product mappings

2009-06-05 Thread Stuart Sierra
On Jun 5, 10:56 am, Daniel Jomphe wrote: > I need to generate a list of all possible American zipcodes, MD5- > digested. Later on, I will need to do much more involving stuff, I'm not sure if this is exactly what you want, but maybe this will help: (defn all-zipcodes [] (for [x (range 1 9

Re: Optimizing cross-product mappings

2009-06-05 Thread Daniel Lyons
On Jun 5, 2009, at 8:56 AM, Daniel Jomphe wrote: > I need to generate a list of all possible American zipcodes, MD5- > digested. Later on, I will need to do much more involving stuff, > processor-wize, with this. But already, generating a naive list of all > possible zipcodes is taking quite a d

Re: Optimizing cross-product mappings

2009-06-05 Thread Sean Devlin
Try adding type hints. Assuming all-zips returns a list of strings: (defn all-zips-MD5 [] "Returns a lazy list of all possible American zipcodes, as MD5 digests." (let [digester (java.security.MessageDigest/getInstance "MD5")] (map (fn [#^java.util.String to-digest] (.update

Optimizing cross-product mappings

2009-06-05 Thread Daniel Jomphe
I need to generate a list of all possible American zipcodes, MD5- digested. Later on, I will need to do much more involving stuff, processor-wize, with this. But already, generating a naive list of all possible zipcodes is taking quite a deal of time: user> (time (dorun (take 100 (all-zips)))