> As for the escaped quotes, you may be using pr or prn to print, or maybe
you are
> using pr-str to produce the string representation. I can't be sure.
At the moment I create the string like this:
document-as-string (str "{\"transaction-id\" : \"" transaction-id
"\", \"message\" : \"" message "\"}")
document-as-byte-array (bytes (byte-array (map (comp byte int)
document-as-string)))
The crazy thing is that this sometimes works, but other times the quote
marks appear in Redis as escaped quote marks. As near as I can tell, the
important factor is the length of the string. A short string is likely to
have its quote marks escaped. A long string does not have its quote marks
escaped.
My co-worker is working on the Java app. I am working on the Clojure app.
We can both adjust out apps freely, just so long as we can get data to and
from each other, in a manner that allows us to eventually cast the data to
and from JSON.
Any suggestions are welcome.
On Wednesday, July 8, 2015 at 8:58:53 PM UTC-4, Francis Avila wrote:
>
> Who is saving these strings, and who is reading them? Do you have complete
> control over both apps, or does one of them need to be aligned with the
> other?
>
> If the Java app is the baseline, you need to know the exact details of the
> format of the data it saves. Just knowing "it's JSON" is not enough,
> because there may be other data types (e.g. dates) that don't have native
> JSON representations. (The go-to JSON de/encoder for clojure is cheshire:
> https://github.com/dakrone/cheshire )
>
> I took a quick look at Jedis and the easy, default way of using it is with
> strings. It will encode strings to UTF-8 before sending to Redis and decode
> from UTF-8 on read. You can set raw byte arrays too (which will not be
> altered in any way before sending), but it's not clear to me how it can
> read out raw byte arrays. (I'm sure there's a way, but it's not immediately
> obvious.)
>
> As for the escaped quotes, you may be using pr or prn to print, or maybe
> you are using pr-str to produce the string representation. I can't be sure.
>
> On Wednesday, July 8, 2015 at 5:31:27 PM UTC-5, gingers...@gmail.com
> wrote:
>>
>> And I have another stupid question. Using the above code, I am sometimes
>> getting strings in Redis that have escaped quotation marks, like this:
>>
>> " \"transaction-id\" : \" 1ec47c2e-21ee-427c-841c-80a0f89f55d7 \"
>> \"debrief\" : \" Susan Hilly at Citi called to get a quotation for
>> discounted weekly car rental for approximately 3 cars per week, or 150
>> rentals annually. \" "
>>
>> Why is that happening?
>>
>>
>>
>> On Wednesday, July 8, 2015 at 5:38:20 PM UTC-4, gingers...@gmail.com
>> wrote:
>>>
>>> Francis Avila,
>>>
>>> Thank you for your response. The Java app is using Jedis and the Clojure
>>> app is using Carmine. I'm wondering if you can suggest what you think would
>>> be the easiest way to allow these 2 apps to understand each other's strings?
>>>
>>> You were correct about how unsafe the above code was. I tested it for
>>> less than 15 minutes and ran into the fact that a \n newline made a mess of
>>> everything.
>>>
>>>
>>>
>>> On Wednesday, July 8, 2015 at 5:15:35 PM UTC-4, Francis Avila wrote:
>>>>
>>>> You are running into Carmine's automatic nippy serialization.
>>>> https://github.com/ptaoussanis/carmine#serialization
>>>>
>>>> Redis only stores byte arrays (what it calls "strings"). Carmine uses
>>>> the nippy library (the meaning of "NPY" in your byte stream) to represent
>>>> rich types compactly as bytes. https://github.com/ptaoussanis/nippy
>>>>
>>>> If you give Carmine a byte array to store, it will store it directly
>>>> without nippy-encoding it. E.g. (.getBytes "{}" "UTF-8")
>>>>
>>>> BTW your document-as-string example is extremely unsafe: how will you
>>>> reliably read this message out again? e.g. what if the 'debrief' string
>>>> contains a single quote? Use a proper serialization format.
>>>>
>>>> So the key is to have both your Clojure and Java app store *bytes* in
>>>> Redis using the same serialization. You can store anything you want
>>>> (nippy,
>>>> utf-8-encoded json, fressian, bson, utf-8 xml, utf-16 java strings,
>>>> whatever) as long as it's bytes and it's read and written the same way in
>>>> all your apps.
>>>>
>>>> The Redis library your Java app is using may have its own automatic
>>>> de/serialization, too. You need to find out what it's doing and either
>>>> work
>>>> with this or turn it off, just like with Carmine.
>>>>
>>>> Nippy unfortunately does not have a Java API out of the box:
>>>> https://github.com/ptaoussanis/nippy/issues/66
>>>>
>>>>
>>>> On Wednesday, July 8, 2015 at 3:35:49 PM UTC-5, gingers...@gmail.com
>>>> wrote:
>>>>>
>>>>>
>>>>> I am not sure if this is a Clojure question or a Java question. I
>>>>> don't know Java, so I could use whatever help folks can offer.
>>>>>
>>>>> "primitive string" here means what I can write when I am at the
>>>>> terminal.
>>>>>
>>>>> We have 2 apps, one in Clojure, one in Java. They talk to each other
>>>>> via Redis. I know the Java app can read stuff out of Redis, using our
>>>>> "transaction-id", if I use the terminal and open up "redis-clj" and write
>>>>> a
>>>>> string directly from the terminal. But I have this Clojure code, which
>>>>> depends on Peter Taoussanis's Carmine library:
>>>>> (defn worker [document]
>>>>> {:pre [(string? (:transaction-id document))]}
>>>>> (let [transaction-id (:transaction-id document)
>>>>> document-as-string (str "{'transaction-id' : '" transaction-id
>>>>> "', 'debrief' : '" (:debrief document) "'}" )
>>>>> redis-connection {:pool {} :spec {:host "127.0.0.1" :port 6379
>>>>> }}]
>>>>> (timbre/log :trace " message we will send to NLP "
>>>>> document-as-string)
>>>>> (carmine/wcar redis-connection (carmine/set transaction-id
>>>>> document))
>>>>> (loop [document-in-redis (carmine/wcar redis-connection
>>>>> (carmine/get transaction-id))]
>>>>>
>>>>> (if-not (.contains (first document-in-redis) "processed")
>>>>> (recur (carmine/wcar redis-connection (carmine/get
>>>>> transaction-id)))
>>>>> (do
>>>>> (carmine/wcar redis-connection (carmine/del transaction-id))
>>>>> document-in-redis)))))
>>>>>
>>>>> This line in particular, I have tried doing this several ways:
>>>>>
>>>>> document-as-string (str "{'transaction-id' : '" transaction-id
>>>>> "', 'debrief' : '" (:debrief document) "'}" )
>>>>>
>>>>> In Redis, I expect to see:
>>>>>
>>>>> {'transaction-id' : '42e574e7-3b80-424a-b9ff-01072f1e0358', 'debrief'
>>>>> : 'Smeek Hallie of Withers, Smeg, Harrington and Norvig responded to our
>>>>> proposal and said his company is read to move forward. The rate of $400
>>>>> per
>>>>> ton of shredded paper was acceptable to them, and they shred about 2 tons
>>>>> of documents every month. $96,000 in potential revenue annually. I will
>>>>> meet with him tomorrow and we will sign the contract.'}
>>>>>
>>>>> But if I then launch redis-cli, I see:
>>>>>
>>>>>
>>>>> 127.0.0.1:6379> keys *
>>>>> 1) "42e574e7-3b80-424a-b9ff-01072f1e0358"
>>>>>
>>>>> 127.0.0.1:6379> get "42e574e7-3b80-424a-b9ff-01072f1e0358"
>>>>> "\x00>NPY\b\x00\x00\x01\xfc\xf1\xfe\x1b\x00\x00\x00\nj\nip-addressi\x0e165.254.84.238j\x05tokeni$46b87d64-cff3-4b8b-895c-e089ac59544dj\x0bapi-versioni\x02v1j\x0etransaction-idi$42e574e7-3b80-424a-b9ff-01072f1e0358j\adebrief\r\x00\x00\x01YSmeek
>>>>>
>>>>> Hallie of Withers, Smeg, Harrington and Norvig responded to our proposal
>>>>> and said his company is rea-\x00\xf1\x06move forward. The
>>>>> raty\x00\xf0\x0c$400 per ton of shredded pa\x16\x00\xf1\bwas acceptable
>>>>> to
>>>>> them,q\x00Bthey0\x00\x80 about 2E\x00\x10sF\x00\xf1Ldocuments every
>>>>> month.
>>>>> $96,000 in potential revenue annually. I will meet with him
>>>>> tomorrow{\x00\"we#\x00@sign\x92\x00\xa0 contract."
>>>>>
>>>>>
>>>>> I don't know what all of those extra characters are. The Java app is
>>>>> not picking this item up, so I assume the Java app is not seeing this as
>>>>> a
>>>>> string. I expected this to look the same as if I had written this at the
>>>>> terminal:
>>>>>
>>>>> {'transaction-id' : '42e574e7-3b80-424a-b9ff-01072f1e0358', 'debrief'
>>>>> : 'Smeek Hallie of Withers, Smeg, Harrington and Norvig responded to our
>>>>> proposal and said his company is read to move forward. The rate of $400
>>>>> per
>>>>> ton of shredded paper was acceptable to them, and they shred about 2 tons
>>>>> of documents every month. $96,000 in potential revenue annually. I will
>>>>> meet with him tomorrow and we will sign the contract.'}
>>>>>
>>>>> I assume it is easy to get a string into a format that can be
>>>>> understood by both a Clojure app and a Java app. I don't care what format
>>>>> that is, but it needs to be consistent.
>>>>>
>>>>> Can anyone make suggestions about what I can do to make sure the
>>>>> Clojure app and the Java app both write to Redis using a format that the
>>>>> other will understand? In particular, both apps need to see the
>>>>> "'transaction-id".
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups
"Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to clojure+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
