Filip
Peter Warren wrote:
Just to make sure that I wasn't crazy, I did some more tests, including using your cometgui.jar. My results still show both read errors and END events. Is it possible that it's a platform issue? I'm running Windows XP Pro SP2. As I mentioned before I'm using the tomcat 6.0.x trunk as of 21-01-2008. Also, I re-started tomcat in between all tests to make sure there weren't stray bytes hanging around somewhere. My server code is the same as shown previously in this thread. Peter Here are my results: 1) Running LastChunkTest (code below) with last chunk of 0crlfcrlf, server shows: event: BEGIN, subtype: null event: READ, subtype: null Read 10 bytes: comet test for session: 627378C9DEE2817A93EBAC190DE47599 read error 2) Running LastChunkTest with last chunk of 0crlfcrlf and a 50 ms sleep before sending last chunk, server shows: event: BEGIN, subtype: null event: READ, subtype: null Read 10 bytes: comet test for session: A00EFFC9A9FE8CBF1833D204EF00667C event: END, subtype: null 3) Running LastChunkTest with a last chunk of 0crlf, server shows: event: BEGIN, subtype: null event: READ, subtype: null Read 10 bytes: comet test for session: DD41C198E3D5CD6E7D73FC0D7B37E86F 4) Running cometgui.jar with 0crlfcrlf: client sends (without the dashes): --- GET /CometTest HTTP/1.1 User-Agent: Filips Awesome Client/1.0 Host: 127.0.0.1:8080 Transfer-Encoding: chunked 10 test data test 1 0 --- server shows: event: BEGIN, subtype: null event: READ, subtype: null Read 16 bytes: test data test 1 for session: 4023C27063C5674C1F37D315C3AF0AFC read error 5) Running cometgui.jar with 0crlf: client sends (without the dashes): --- GET /CometTest HTTP/1.1 User-Agent: Filips Awesome Client/1.0 Host: 127.0.0.1:8080 Transfer-Encoding: chunked 10 test data test 1 0 --- server shows: event: BEGIN, subtype: null event: READ, subtype: null Read 16 bytes: test data test 1 for session: BC5EA8DA373D1E6726020F44F71165C7 side note: Not sure if this is important, but when doing the following steps in cometgui for the client data shown above, I get the exception below: connect submit disconnect Jan 22, 2008 10:42:49 AM org.apache.catalina.core.StandardWrapperValve event SEVERE: Servlet.service() for servlet CometTest threw exception java.io.EOFException: Unexpected EOF read on the socket at org.apache.coyote.http11.InternalNioInputBuffer.readSocket(InternalNi oInputBuffer.java:589) at org.apache.coyote.http11.InternalNioInputBuffer.fill(InternalNioInput Buffer.java:868) at org.apache.coyote.http11.InternalNioInputBuffer$SocketInputBuffer.doR ead(InternalNioInputBuffer.java:892) at org.apache.coyote.http11.filters.ChunkedInputFilter.readBytes(Chunked InputFilter.java:243) at org.apache.coyote.http11.filters.ChunkedInputFilter.parseCRLF(Chunked InputFilter.java:326) at org.apache.coyote.http11.filters.ChunkedInputFilter.parseEndChunk(Chu nkedInputFilter.java:356) at org.apache.coyote.http11.filters.ChunkedInputFilter.doRead(ChunkedInp utFilter.java:136) at org.apache.coyote.http11.InternalNioInputBuffer.doRead(InternalNioInp utBuffer.java:830) at org.apache.coyote.Request.doRead(Request.java:428) at org.apache.catalina.connector.InputBuffer.realReadBytes(InputBuffer.j ava:298) at org.apache.tomcat.util.buf.ByteChunk.substract(ByteChunk.java:405) at org.apache.catalina.connector.InputBuffer.read(InputBuffer.java:313) at org.apache.catalina.connector.CoyoteInputStream.read(CoyoteInputStrea m.java:162) at com.seekspeak.server.debug.CometTestServlet.event(CometTestServlet.ja va:31) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilterEvent (ApplicationFilterChain.java:470) at org.apache.catalina.core.ApplicationFilterChain.doFilterEvent(Applica tionFilterChain.java:363) at org.apache.catalina.core.StandardWrapperValve.event(StandardWrapperVa lve.java:422) at org.apache.catalina.core.StandardContextValve.event(StandardContextVa lve.java:252) at org.apache.catalina.core.StandardHostValve.event(StandardHostValve.ja va:179) at org.apache.catalina.valves.ValveBase.event(ValveBase.java:200) at org.apache.catalina.valves.ValveBase.event(ValveBase.java:200) at org.apache.catalina.core.StandardEngineValve.event(StandardEngineValv e.java:128) at org.apache.catalina.connector.CoyoteAdapter.event(CoyoteAdapter.java: 198) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.jav a:292) at org.apache.coyote.http11.Http11NioProcessor.process(Http11NioProcesso r.java:876) at org.apache.coyote.http11.Http11NioProtocol$Http11ConnectionHandler.pr ocess(Http11NioProtocol.java:719) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoin t.java:2080) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec utor.java:885) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor .java:907) at java.lang.Thread.run(Thread.java:619) event: ERROR, subtype: null Code for LastChunkTest: ======================= package test; import java.io.IOException; import java.io.InputStream; import java.io.OutputStream; import java.net.NoRouteToHostException; import java.net.Socket; import java.net.URL; public class LastChunkTest { private static final String ENCODING = "ISO-8859-1"; private static final String DELIMITER = "\r\n"; private URL url; private InputStream inputStream; private OutputStream outputStream; private Socket socket; public static void main(String[] args) throws Exception { LastChunkTest test = new LastChunkTest(); test.test(); } private void test() throws Exception { url = new URL("http://www.seekspeak.com/CometTest";); initConnection(); sendHeaders(); send("comet test"); // uncomment sleep to get END event instead of read error // try { // Thread.sleep(50); // } catch (InterruptedException ie) { // // do nothing // } sendLastChunk(); // block on read to keep app alive - server never sends response inputStream.read(); } private void initConnection() throws IOException { int port = url.getPort(); port = (port < 0) ? url.getDefaultPort() : port; try { socket = new Socket(url.getHost(), port); socket.setKeepAlive(true); inputStream = socket.getInputStream(); outputStream = socket.getOutputStream(); } catch (NoRouteToHostException nrthe) { System.out.println("host: " + url.getHost()); nrthe.printStackTrace(); } } private void sendHeaders() throws IOException { String path = url.getPath(); StringBuffer outputBuffer = new StringBuffer(); outputBuffer.append("POST " + path + " HTTP/1.1" + DELIMITER); outputBuffer.append("Host: " + url.getHost() + DELIMITER); outputBuffer.append("User-Agent: CometTest" + DELIMITER); outputBuffer.append("Connection: keep-alive" + DELIMITER); outputBuffer.append("Content-Type: text/plain" + DELIMITER); outputBuffer.append("Transfer-Encoding: chunked" + DELIMITER); outputBuffer.append(DELIMITER); byte[] outputBytes = outputBuffer.toString().getBytes(ENCODING); outputStream.write(outputBytes); outputStream.flush(); } private void send(String chunkData) throws IOException { byte[] chunkBytes = chunkData.getBytes(ENCODING); String hexChunkLength = Integer.toHexString(chunkBytes.length); StringBuffer outputBuffer = new StringBuffer(); outputBuffer.append(hexChunkLength); outputBuffer.append(DELIMITER); outputBuffer.append(chunkData); outputBuffer.append(DELIMITER); byte[] outputBytes = outputBuffer.toString().getBytes(ENCODING); outputStream.write(outputBytes); outputStream.flush(); } private void sendLastChunk() throws IOException { byte[] outputBytes = new String("0" + DELIMITER).getBytes(ENCODING); outputStream.write(outputBytes); outputStream.flush(); } } On Jan 22, 2008 9:07 AM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote:I still don't get the END event, however, thanks for pointing it out, you did find a regression bug about the timeout http://svn.apache.org/viewvc?view=rev&revision=614249 I've added the patch above to the proposal list for 6.0.x to fix the timeout issue Filip Peter Warren wrote:as I mentioned, the "last chunk" doesn't generate an END event, I tried it locally. of course against 6.0.x trunk.I played around a bit because I was definitely getting an END event and found: Sending 0crlf does not generate and END event. However sending 0crlfcrlf, which is what HttpURLConnection does, does generate an END (or sometimes a read error - see below...) Looking at the http spec, it seems like 0crlfcrlf is actually the proper way to terminate the chunk body: Chunked-Body = *chunk last-chunk trailer CRLF last-chunk = 1*("0") [ chunk-extension ] CRLF Am I reading that correctly? Note about END and read error: When running both the client and the server locally (i.e. little latency), sending 0crlfcrlf would sometimes generate a read error (i.e. inputStream.isAvailable() > 0 would be true and then number of bytes read would be < 0) and sometimes an END event. I tried with both sockets and HttpURLConnection and saw similar behavior. However when using HttpURLConnection I could add a delay of 50 millis. and guarantee that I always got an end event (see code below). I am using the latest 6.0.x trunk updated locally today. Peter CLIENT CODE =========== URL url = new URL("http://www.seekspeak.com/CometTest";); HttpURLConnection urlConn = (HttpURLConnection) url.openConnection(); urlConn.setRequestMethod("POST"); urlConn.setChunkedStreamingMode(-1); // use default chunk length urlConn.setReadTimeout(0); urlConn.setDoInput(true); urlConn.setDoOutput(true); urlConn.connect(); PrintWriter out = new PrintWriter(urlConn.getOutputStream(), true); out.print("test"); out.flush(); try { // sleep to guarantee an END event - remove this sleep to get read error Thread.sleep(50); } catch (InterruptedException ie) { // do nothing } urlConn.getInputStream(); COMET SERVLET CODE: ============= public void event(CometEvent cometEvent) throws IOException, ServletException { System.out.println("event: " + cometEvent.getEventType() + ", subtype: " + cometEvent.getEventSubType()); if (cometEvent.getEventType() == CometEvent.EventType.ERROR) { cometEvent.close(); } else if (cometEvent.getEventType() == CometEvent.EventType.END) { cometEvent.close(); } else if (cometEvent.getEventType() == CometEvent.EventType.READ) { HttpServletRequest request = cometEvent.getHttpServletRequest(); InputStream inputStream = request.getInputStream(); byte[] buf = new byte[512]; do { int n = inputStream.read(buf); // can throw an IOException if (n > 0) { System.out.println("Read " + n + " bytes: " + new String(buf, 0, n) + " for session: " + request.getSession(true).getId()); } else if (n < 0) { System.out.println("read error"); return; } } while (inputStream.available() > 0); } } On Jan 21, 2008 11:53 AM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote:answers inline Peter Warren wrote:First off, thanks for your responses. The contributors to this list are extremely responsive, patient, and helpful, and I really appreciate it! Hmm, in your test case did you set the HttpURLConnection to use chunked transfers (setChunkedStreamingMode(...))? I find if I use chunked transfers, the HttpURLConnection sends a "last chunk" message upon reading input from the server, which generates a comet END event on the server. If I don't use chunked transfers, no END event is generated because no "last chunk" is sent by the client.as I mentioned, the "last chunk" doesn't generate an END event, I tried it locally. of course against 6.0.x trunk.Which brings up an option I never considered: will a comet servlet function properly with non-chunked transfer (i.e. no transfer-coding header)? It seems to.yes, it can, just send a very large content-length headerLastly, I'm still a little confused about requiring the comet event to be closed on an END event. Doesn't this mean that tomcat comet can't handle pipelined requests? If a client sends 2 pipelined requests, it will send a "last chunk" to indicate the end of the first request. This "last chunk" will generate an END event on the server, which then requires the connection to be closed. After the comet event is closed the server cannot send a response to the client.pipelined requests are not defined by the HTTP spec for POST methods, only for GET assuming the pipelining you are talking about is true pipelining :) if you just mean, next request, then yes, tomcat handles that just fine, and that is why you have to call event.close()The END event docs indicate that pipelined request will generate an END event: ...End will also be called when data is available and the end of file is reached on the request input (this usually indicates the client has pipelined a request).depends on what you mean by pipeline, see aboveThanks, Peter On Jan 20, 2008 8:15 PM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote:now I get it. I just ran through a test case, and an END event was not thrown just because there was an end chunk. the response is very much still open at that point Filip Peter Warren wrote:What java.net.HttpURLConnection has to do with Tomcat and comet is that HttpURLConnection is Java's implementation of an http client and will likely be used by people developing comet apps for Tomcat. In my case, I want to use it because I can't use raw sockets on my applet client due to permission problems when trying to use sockets behind a proxy. I understand that asynchronous writes are possible, but they're not when using HttpURLConnection because HttpURLConnection sends a "last chunk" message when it's done with its request. "Last chunk" generates a comet end event, which then requires that the connection to the client be closed. I guess I don't understand why tomcat needs to close the connection after an END event. It seems to me that the "last chunk" message from the client simply indicates that the client is done sending its request. Why does the server need to close the connection when the client finishes its request? Peter On Jan 19, 2008 6:01 PM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote:I'm not sure what HttpURLConnection has to do with Tomcat or comet. and yes, asynchronous writes are possible, just not after the END or ERROR events have been issued Filip Peter Warren wrote:Does that mean that HttpURLConnection cannot be used for comet requests with asynchronous (i.e. delayed) responses? It would seem so to me since HttpURLConnection always sends an END message before reading from the server and since the server can no longer write to the client after closing the comet event. Am I missing something? Is there a way to write to the client after the comet event is closed? Would you consider it a bug that HttpURLConnection is implemented that way? Peter On Jan 18, 2008 9:21 PM, Filip Hanik - Dev Lists <[EMAIL PROTECTED]> wrote:during end and error, you MUST close the Comet event Filip Peter Warren wrote:What do I do to make the END event stop repeating? I don't want to close the CometEvent yet because the server is waiting for data to send to the client. If I don't close the comet event, the END event repeats incessantly. I'm using an unsigned applet as a comet client. To accommodate proxies, I've had to change my comet client to use HttpURLConnection instead of Sockets. (Accessing ProxySelector from an applet to create a socket with a proxy generates an AccessControlException.) HttpURLConnection unfortunately sends a 0crlf when its input stream is retrieved for reading. This generates a Comet END event. Short of closing the comet event, how can I make the server stop notifying me of END events? I can't close the comet event because I want to hold onto the comet output stream for use later to send data to the client. >From the comet docs: EventType.END: End may be called to end the processing of the request. Fields that have been initialized in the begin method should be reset. After this event has been processed, the request and response objects, as well as all their dependent objects will be recycled and used to process other requests. End will also be called when data is available and the end of file is reached on the request input (this usually indicates the client has pipelined a request). This seems to indicate that even if I could get the END event to go away quietly, the comet event's output stream might no longer be usable anyway. It seems to me I have 3 options: 1) figure out how to make the comet END event stop repeating and hope it's output stream still works 2) figure out how to keep HttpURLConnection from sending 0crlf (don't know if that can be done) 2) use sockets with ProxySelector (which requires signing my applet and getting users to grant it privileges) Thanks, Peter --------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------------------------------------------------- To start a new topic, e-mail: users@tomcat.apache.org To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
