Hi Travis,
connection-read appears to be subtlety wrong.
If all the bytes are ready to be read then it is correct, but if only
some are read it is incorrect. Now in the large data scenario the
latter will appear if there are no delays. Inserting a sleep delays
the program per packet such that there is always data to be read -
causing it to work. This is a much more likely scenario...
How is it wrong?
the otherwise read more data part recurs until new-bytes-read == -1
(connection closed) so its no longer reading the amount specified by
the packet header. I think this was unintentional. A better solution
would be to just do a blocking read instead which can only give you
the right number of bytes or error. Otherwise the maths for remaining
data needs to be fixed. (Blocking is simpler and has the same result
and doesn't put you in a CPU spin).
Also regarding the state transitions, it looks to me that the way it
is set up now there has to be an eof to transition to :row-start
state, which will lead to another recur to reading the (presumably
finished) response which will be eof also. However I can't see how
that would result in your described issue its probably me just
misunderstanding. Your issue sounds highly related to a 'message'
being split across multiple packets - as it is dependent on data
size... so I believe the real problem is the way read is handled if
all the bytes aren't ready to be read.
I've had to make some assumptions about what you are doing so forgive
me if no relevant.
Regards,
Tim.
On Jun 1, 1:02 pm, tmountain <tinymount...@gmail.com> wrote:
> Hi all - I'm in the process of writing a proxy for MySQL in Clojure,
> and until now everything has been going smoothly. My project has
> reached the point where it can shuffle data up and down the wire
> between client and server with accurate results, but I'm running into
> a strange issue. I've noticed that when a certain result size is
> reached, (exactly 138 rows in this case), I get a malformed packet
> response from the client.
>
> Naturally, I assumed this was just an issue with mishandling the data
> being sent to and fro somehow, but then I ran into something strange.
> In trying to nail down the issue, I wrote a debug macro which inserts
> prints a given statement so long as a *debug* variable is set to true.
> As soon as I enabled the macro, the problem resolved itself, and the
> client returned the appropriate result. After a brief WTF moment, I
> decided it might be some kind of timing issue and changed the debug
> macro to simply do a Thread.sleep for 100 milliseconds. Again, this
> resolved the problem. Finally, I narrowed down the offending code to
> the following:
>
> (defn get-result-query [client server state]
> "handle the states necessary to return a standard query result"
> (let [server-response (connection-read server)]
> (Thread/sleep 100) ; !!!!!!!!! remove this, and the function will
> sporadically fail !!!!!!!!!
> (connection-write client server-response)
> (cond (and (= state :start) (eof-packet? server-response))
> (recur client server :row-start)
> (and (= state :row-start) (eof-packet? server-response))
> {:state :send-command :server server}
> (error-packet? server-response)
> {:state :send-command :server server}
> :else (recur client server state))))
>
> It seems that reading from the socket without a subsequent pause
> introduces problems for some reason. I have no idea why this would
> happen unless the data isn't fully down the wire or something along
> those lines. For good measure, here's the connection-read function as
> well:
>
> (defn connection-read [#^Socket conn]
> "reads data up to the amount specified by the packet header"
> (let [#^InputStream instream (. conn (getInputStream))
> bytes (make-array (. Byte TYPE) 4)]
> (. instream (read bytes 0 4))
> ; make an array big enough for header + data
> (let [data (make-array (. Byte TYPE)
> (+ 4 (packet-length? bytes)))]
> (System/arraycopy bytes 0 data 0 4)
> ; read data into the buffer
> (loop [bytes-read (. instream (read data 4 (- (alength data)
> 4)))]
> ; if we got all the desired data in one pass, return it
> (if (>= bytes-read (- (alength data) 4))
> data
> ; otherwise, read more data
> (let [new-bytes-read (. instream
> (read data
> bytes-read
> (- (alength data) bytes-
> read)))]
> (if (== new-bytes-read -1)
> data
> (recur (+ bytes-read new-bytes-read)))))))))
>
> If anybody has any suggestions to why this would be happening, I'd
> really appreciate it. I intend to release this project OSS when it's
> fully functional, and this is one of the few roadblocks in my way.
>
> Thanks,
> Travis
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clojure@googlegroups.com
Note that posts from new members are moderated - please be patient with your
first post.
To unsubscribe from this group, send email to
clojure+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
-~----------~----~----~----~------~----~------~--~---
