Parameters disappear from PUTs
I am submitting forms to a restful interface using an HTTP PUT with the params in the header of the PUT. I believe that's normal, and it works just fine on Jetty. The params are definitely sent. When I load my app into Tomcat 6 (Ubuntu), the form submission works perfectly if I use a POST: the params are definitely received. If I use a PUT, it works, but the parameters are missing -- not blank, but gone completely. I can also use a DELETE and that works (no params involved, of course), so it's not that PUT/DELETE are blocked. I've searched the docs, the wiki, and this list, but can't find anything about this. Is this expected behavior? Is there a setting somewhere? I found a setting in web.xml, but that appeared to refer only to SSI and was an all or nothing switch. Here, the PUT goes through and all the right methods are called, resulting in a row in the database. But the row is blank. (The only required field is the ID and that's passed in the URL.) I'm stumped. Anyone have any ideas? TIA, Chas. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Parameters disappear from PUTs
>> From: c...@munat.com [mailto:c...@munat.com] >> Subject: Parameters disappear from PUTs >> >> When I load my app into Tomcat 6 (Ubuntu), the form submission works >> perfectly if I use a POST: the params are definitely received. If I >> use a PUT, it works, but the parameters are missing > > 1) Tell us your /exact/ Tomcat aversion, and whether it's a 3rd-party > repackaged one or a real Tomcat from tomcat.apache.org. (The Ubuntu > flavor of Tomcat gives many people grief, albeit not with the symptoms you > describe.) I don't actually have a Tomcat aversion, which is why I'm using it. (It's more of an aversion to computers in general, I'd say.) But the Tomcat version is 6.0.20-2ubuntu2. > 2) Tell us the /exact/ JVM version and vendor you're using. Looks like Sun JDK 6-15-1. > 3) Take a Wireshark or equivalent trace to verify that the parameters are > being sent over the wire. Wireshark is kind of a bear to use and I'm not at the server. If I used Live HTTP Header in Firefox and confirmed that the params are going out, is that enough? Because I did and they are. The only difference in the headers is that the POST header includes: Pragma: no-cache Cache-Control: no-cache > 4) Enable the Request Dumper Valve to see if the parameters are making it > into Tomcat (but heed the warning about character encoding side effects): > http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html#Request%20Dumper%20Valve I did. If I POST the params are there. If I PUT, they are not, repeat, not. > 5) Try the latest Tomcat 6.0.24 download and see if the problem persists. > Note the following bug report: > https://issues.apache.org/bugzilla/show_bug.cgi?id=37794 > which was corrected in 6.0.21. It's for handling POST parameters, but > might be related. Sadly, this is not currently possible. Apt-get is still at 20, and nothing goes on the server that doesn't go through Apt-get. Fortunately, I can work around it by using POST for now, but it would be nice to be able to do REST right. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Parameters disappear from PUTs
OK, turns out my brother has wireshark installed. We ran it, and the packets are definitely getting to the server with the PUT params intact. So they arrive at the server, but they are missing from the output of the valve. Does this mean Tomcat is hosed? I thought maybe I'd test 24 on my laptop, but macports is still at 20, too. Guess I'll have to do it the hard way. >>> From: c...@munat.com [mailto:c...@munat.com] >>> Subject: Parameters disappear from PUTs >>> >>> When I load my app into Tomcat 6 (Ubuntu), the form submission works >>> perfectly if I use a POST: the params are definitely received. If I >>> use a PUT, it works, but the parameters are missing >> >> 1) Tell us your /exact/ Tomcat aversion, and whether it's a 3rd-party >> repackaged one or a real Tomcat from tomcat.apache.org. (The Ubuntu >> flavor of Tomcat gives many people grief, albeit not with the symptoms >> you >> describe.) > > I don't actually have a Tomcat aversion, which is why I'm using it. (It's > more of an aversion to computers in general, I'd say.) But the Tomcat > version is 6.0.20-2ubuntu2. > >> 2) Tell us the /exact/ JVM version and vendor you're using. > > Looks like Sun JDK 6-15-1. > >> 3) Take a Wireshark or equivalent trace to verify that the parameters >> are >> being sent over the wire. > > Wireshark is kind of a bear to use and I'm not at the server. If I used > Live HTTP Header in Firefox and confirmed that the params are going out, > is that enough? Because I did and they are. The only difference in the > headers is that the POST header includes: > > Pragma: no-cache > Cache-Control: no-cache > > >> 4) Enable the Request Dumper Valve to see if the parameters are making >> it >> into Tomcat (but heed the warning about character encoding side >> effects): >> http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html#Request%20Dumper%20Valve > > I did. If I POST the params are there. If I PUT, they are not, repeat, > not. > >> 5) Try the latest Tomcat 6.0.24 download and see if the problem >> persists. >> Note the following bug report: >> https://issues.apache.org/bugzilla/show_bug.cgi?id=37794 >> which was corrected in 6.0.21. It's for handling POST parameters, but >> might be related. > > Sadly, this is not currently possible. Apt-get is still at 20, and nothing > goes on the server that doesn't go through Apt-get. Fortunately, I can > work around it by using POST for now, but it would be nice to be able to > do REST right. > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
RE: Parameters disappear from PUTs
Sorry, I checked and it is already "no". I have full control of the server, albeit it's 2000 miles away... Chas. > As a quick check, can you go into /etc/init.d/tomcat6 and at around line > 68 you'll see: > > TOMCAT_SECURITY=yes > > If you're allowed to, change that line to: > > TOMCAT_SECURITY=no > > Then restart tomcat with: > > ../tomcat6 restart > > Of course, you have to be root to do this. > > On first glance, I don't see a security policy that would cause this. > However, if you look at the Ubuntu bugs database, there are many instances > of people complaining about the security manager being enabled by default. > > Just my two cents . . . > > /mde/ > > --- On Wed, 2/3/10, c...@munat.com wrote: > >> From: c...@munat.com >> Subject: RE: Parameters disappear from PUTs >> To: "Tomcat Users List" >> Date: Wednesday, February 3, 2010, 10:43 PM >> OK, turns out my brother has >> wireshark installed. We ran it, and the >> packets are definitely getting to the server with the PUT >> params intact. >> So they arrive at the server, but they are missing from the >> output of the >> valve. Does this mean Tomcat is hosed? >> >> I thought maybe I'd test 24 on my laptop, but macports is >> still at 20, >> too. Guess I'll have to do it the hard way. >> >> >>> From: c...@munat.com >> [mailto:c...@munat.com] >> >>> Subject: Parameters disappear from PUTs >> >>> >> >>> When I load my app into Tomcat 6 (Ubuntu), the >> form submission works >> >>> perfectly if I use a POST: the params are >> definitely received. If I >> >>> use a PUT, it works, but the parameters are >> missing >> >> >> >> 1) Tell us your /exact/ Tomcat aversion, and >> whether it's a 3rd-party >> >> repackaged one or a real Tomcat from >> tomcat.apache..org. (The Ubuntu >> >> flavor of Tomcat gives many people grief, albeit >> not with the symptoms >> >> you >> >> describe.) >> > >> > I don't actually have a Tomcat aversion, which is why >> I'm using it. (It's >> > more of an aversion to computers in general, I'd say.) >> But the Tomcat >> > version is 6.0.20-2ubuntu2. >> > >> >> 2) Tell us the /exact/ JVM version and vendor >> you're using. >> > >> > Looks like Sun JDK 6-15-1.. >> > >> >> 3) Take a Wireshark or equivalent trace to verify >> that the parameters >> >> are >> >> being sent over the wire. >> > >> > Wireshark is kind of a bear to use and I'm not at the >> server. If I used >> > Live HTTP Header in Firefox and confirmed that the >> params are going out, >> > is that enough? Because I did and they are. The only >> difference in the >> > headers is that the POST header includes: >> > >> > Pragma: no-cache >> > Cache-Control: no-cache >> > >> > >> >> 4) Enable the Request Dumper Valve to see if the >> parameters are making >> >> it >> >> into Tomcat (but heed the warning about character >> encoding side >> >> effects): >> >> http://tomcat.apache.org/tomcat-6.0-doc/config/valve.html#Request%20Dumper%20Valve >> > >> > I did. If I POST the params are there. If I PUT, they >> are not, repeat, >> > not. >> > >> >> 5) Try the latest Tomcat 6.0.24 download and see >> if the problem >> >> persists. >> >> Note the following bug report: >> >> https://issues.apache.org/bugzilla/show_bug.cgi?id=37794 >> >> which was corrected in 6.0.21. It's for >> handling POST parameters, but >> >> might be related. >> > >> > Sadly, this is not currently possible. Apt-get is >> still at 20, and nothing >> > goes on the server that doesn't go through Apt-get. >> Fortunately, I can >> > work around it by using POST for now, but it would be >> nice to be able to >> > do REST right. >> > >> > >> > >> > >> > >> - >> > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> > For additional commands, e-mail: users-h...@tomcat.apache.org >> > >> > >> >> >> >> - >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >> > > > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [Fwd: Re: Parameters disappear from PUTs]
Re PUT, a little research will show that this has been a hot topic for several years now, at least since Elliotte Rusty Harold's controversial "Why REST Failed" post in 2006. RFC 2616 is quite clear on this topic, IMO. The POST method is used "to request that the origin server accept the *entity* enclosed in the request as a new ***subordinate*** of the resource identified by the Request-URI in the Request-Line." (Emphasis mine.) In the context of a REST web service, this means that if I, for example, wrap a new "Person" entity in a POST request and send it to: example.com/persons then this Person will be added as a subordinate of "persons" -- that is, the URI identifies the group of persons to which this new Person will be added. It does not identify a specific Person. The application will then determine a URI to identify this newly created Person. Note also that this transaction is neither safe nor idempotent, i.e., it makes changes on the server for which the user (I) may be held responsible, and multiple requests result have the side effect of creating multiple identical Persons in the Persons group, each with a different URI -- probably not what I intended. In contrast, the PUT method "requests that the enclosed *entity* be stored *under the supplied Request-URI*." (Emphasis mine, again. Note that both POST and PUT are dealing with entities, but that while POST treats the entity as a subordinate of whatever exists at the Request-URI, PUT identifies the entity *directly* with the Request-URI. So, presuming that I was using a GUID as an identifier, I might send my Person object via PUT to: example.com/persons/6cd79223-3585-4618-ab78-1c20db09c535 This will, if no Person already exists at that address, create a new Person, associate it with that URI, and send a 201 "Created" response. If a Person already exists at that address, the enclosed Person entity replaces that Person, and a 200 "Sure, whatever" response is sent. Note that this transaction is still unsafe, but now it is idempotent: I can repeat the same request again and again, but I still end up with that person at that URI. Thus, my interest in PUT. First, you can't really, at least IMO, implement REST without it. Second, as a functional programmer who likes immutability and the absence of side effects, I find idempotency pretty irresistible. While POST is acceptable for creating new entities (if you don't mind side effects), it is just plain wrong for updating entities. That's what PUT is for! But PUT also allows for creating new entities, so if the URI can be determined by the client -- and GUIDs make this drop dead simple -- then creates can be idempotent as well. So my REST service works thusly: GET/persons -> returns all persons, 200 PUT/persons/ -> creates person , returns 201 PUT/persons/ -> replaces person , returns 200 DELETE /persons/ -> deletes person , returns 204 And everything is idempotent. Except it doesn't work at all if Tomcat strips my entity out of the PUT! With regard to your initial argument re parameters in the header, note that Section 9.6 says that the PUT method "requests that the enclosed ENTITY be stored..." (emphasis mine). A quick check at Section 7 shows that "[a]n entity consists of entity-header fields and an entity-body, although some responses will only include the entity-headers." This would be the same "Entity" to which Section 9.5 on the POST method refers. Hence, Section 9.6 certainly makes clear in its reference to an "entity" that it could be identical to that sent via a POST method. Unless you can point to some text in the RFC that explains that "entity" means something different when applied to PUT vs. POST? Where exactly does it state or even imply that a PUT entity does not include parameters in the header, while a POST entity does? So for Tomcat to unilaterally strip the entity-headers from a PUT request while passing only the optional entity-body seems just, well, wrong to me. For this reason, I'm guessing it's a bug. Chas. > Hi. > > I'll forward your reply to the list, since apparently you did not. > Let me just point out that I was only trying to help, and I do not > believe that there were any grounds in my reply for the sarcastic tone > used in yours. > > In the meantime, I did re-read both sections of the RFC 2616 (9.5 POST > and 9.6 PUT), and I suggest that you do the same, with an open mind. > I agree that neither one of them explicitly talks about "request > parameters". But this paragraph, to me, is rather explicit as to the > difference in /meaning/ between a POST and a PUT : > > quote > The fundamental difference between the POST and PUT requests is > reflected in the different meaning
RE: [Fwd: Re: Parameters disappear from PUTs]
Sorry, but no. There is no ambiguity, unless you try really hard to introduce one with thesaurus firmly in hand. Read the entire section before you comment! >From Section 9.6: "The fundamental difference between the POST and PUT requests is reflected in the different meaning of the Request-URI. The URI in a POST request identifies the resource that will handle the enclosed entity. That resource might be a data-accepting process, a gateway to some other protocol, or a separate entity that accepts annotations. In contrast, the URI in a PUT request identifies the entity enclosed with the request -- the user agent knows what URI is intended and the server MUST NOT attempt to apply the request to some other resource." Clearly, the URI in a POST does *not* represent the entity -- the entity is subordinate to it. Just as clearly, "the URI in a PUT request identifies the entity enclosed with the request." Where is the ambiguity here? Whether that entity is stored on the server in memory, in a individual file on disk, as a section of a file, as a row in a database, or even as an individual cell in a database table, is irrelevant as long as the *enclosed entity* is represented uniquely by the Request-URI, the PUT is satisfied. There is a one-to-one correspondence. If "under" meant "subordinate," then subordinate to *what*? It's not necessary to "suspect that [my] interpretation does correspond to the author's intent" -- their intent is spelled out clearly. (But then, their intent is irrelevant. The RFC is what it is. If it's wrong, then you write a new one. You don't go around trying to guess what the authors' "intent" was -- down that path lies madness. Phillip Hallam-Baker *meant* referrer, but he wrote REFERER and it has been REFERER ever since. You go with the standard as written.) These sorts of discussions are not only pointless, they're potentially harmful as they introduce controversy where none is necessary, wasting time and fraying tempers. I repeat, the RFC is *quite clear* on the difference between POST and PUT -- Roy Fielding, et al, did an excellent job of it. Can we, finally, let it rest there? (Pun intended.) Chas. >> From: c...@munat.com [mailto:c...@munat.com] >> Subject: Re: [Fwd: Re: Parameters disappear from PUTs] >> >> RFC 2616 is quite clear on this topic, IMO. > > Maybe not. > >> The POST method is used "to request that the origin server accept the >> *entity* enclosed in the request as a new ***subordinate*** of the >> resource identified by the Request-URI in the Request-Line." >> >> In contrast, the PUT method "requests that the enclosed *entity* be >> stored *under the supplied Request-URI*." > > Unfortunately, one of the synonyms of "subordinate" is "under", so there's > a degree of ambiguity here. One could interpret the two clauses as being > identical, depending on one's semantic education. (I suspect your > interpretation does correspond to the authors' intent, but it's certainly > not 100% clear.) > > - Chuck > > > THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY > MATERIAL and is thus for use only by the intended recipient. If you > received this in error, please contact the sender and delete the e-mail > and its attachments from all computers. > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [Fwd: Re: Parameters disappear from PUTs]
> Could you please be clear when you say "entity headers": do you mean > HTTP headers? I have a Tomcat 6.0.20 install right in front of me that > definitely does allow the servlet (in my case, a simple JSP that dumps > requests) to read the HTTP headers. HTTP GET parameters are also > correctly provided to the servlet. We were discussing RFC 2616. Entity-headers and entity-body are terms directly from the RFC and are defined therein. No, they are not the same as HTTP headers, though they are contained in the HTTP headers: 7.1 Entity Header Fields Entity-header fields define metainformation about the entity-body or, if no body is present, about the resource identified by the request. Some of this metainformation is OPTIONAL; some might be REQUIRED by portions of this specification. entity-header = Allow; Section 14.7 | Content-Encoding ; Section 14.11 | Content-Language ; Section 14.12 | Content-Length ; Section 14.13 | Content-Location ; Section 14.14 | Content-MD5 ; Section 14.15 | Content-Range; Section 14.16 | Content-Type ; Section 14.17 | Expires ; Section 14.21 | Last-Modified; Section 14.29 | extension-header extension-header = message-header The extension-header mechanism allows additional entity-header fields to be defined without changing the protocol, but these fields cannot be assumed to be recognizable by the recipient. Unrecognized header fields SHOULD be ignored by the recipient and MUST be forwarded by transparent proxies. > >> example.com/persons/6cd79223-3585-4618-ab78-1c20db09c535 > > I didn't check to see if the path info is available for PUT requests. > For that, I'll have to write a separate servlet. I'm willing to write a > test if this is the behavior you are experiencing. > > So, please, what is it that Tomcat is apparently hiding from your servlet? Here are the headers for a PUT request which includes form parameters. They are the string immediately below the cookie information, beginning with "emailAddress": http://localhost:12344/json/members/1b35d32f-714d-4393-b8c2-b4805e0c7a13 PUT /json/members/1b35d32f-714d-4393-b8c2-b4805e0c7a13 HTTP/1.1 Host: localhost:12344 User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-us,en;q=0.5 Accept-Encoding: gzip,deflate Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Keep-Alive: 300 Connection: keep-alive X-Requested-With: XMLHttpRequest Content-Type: application/x-www-form-urlencoded; charset=UTF-8 Referer: http://localhost:12344/ Content-Length: 297 Cookie: JSESSIONID=dexcmg3b1r45 emailAddress=bozo%40clown.com&title=Head%20Clown&nameGiven=Bozo&namesMiddle=The&nameFamily=Clown&namePreferred=Bozo&gender=Male&password=&passwordConfirm=&id=1b35d32f-714d-4393-b8c2-b4805e0c7a13&facebookName=bozo.the.clown&twitterName=i.am.bozo&flickrName=bozos.circus&linkedinName=mr.clown.to.you HTTP/1.x 201 Created Content-Length: 82 Content-Type: text/plain; charset=UTF-8 X-Lift-Version: 1.1-M4 Server: Jetty(6.1.22) As requested by a previous poster, I used Wireshark to examine the packets as they arrived at the server and indeed the form parameters were included in the packets. I also added a Request Dumper Valve to Tomcat and parsed the logs to find the content of the Request. The HTTP headers were there, minus the line with the parameters. I can find no justification in RFC 2616 for stripping this line out -- it is part of the entity-headers as mentioned above, and the RFC also discusses transparency, stating that headers not included in the standard list should be passed through. On a POST request, which is virtually identical to the above, they appear in the request dump. I may be wrong, but it seems to me that Tomcat is stripping those headers out, thus forcing me to use POST even though the correct method should be PUT for what I'm intending. Chas. > > - -chris > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.10 (MingW32) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAktsnUMACgkQ9CaO5/Lv0PBmaACghka+Yd/kqKmhgKcuaLRQED6q > FvQAn3/6JsutlG4NEEYf8u0sCThfO04w > =7yzW > -END PGP SIGNATURE- > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org &
Re: [Fwd: Re: Parameters disappear from PUTs]
REST -- REpresentational State Transfer -- is an architectural style that was developed by Roy Fielding with HTTP, but which is independent of HTTP. No changes need to be made to HTTP to implement REST -- you just need to implement HTTP correctly. There's a pretty good article on Wikipedia about REST, and you can always read Fielding's dissertation. The issue here has nothing to do with REST. I was answering your question about why I'm insisting on PUT rather than accepting POST as most people do. The problem with Tomcat -- if it is indeed a problem with Tomcat -- is that it is not implementing the HTTP 1.1 standard correctly. That's either true or false, but it is unrelated to my desire to use Tomcat to host a RESTful web service. I am neither a Tomcat nor a Java developer, so a patch is not likely to come from me, unless you want it in Scala. I can live with the POST workaround for the moment. But if it is a bug in Tomcat, then it would be nice if someone fixed it. I know several people who would be equally concerned. As for confirming Christopher's point, I'm not sure what you're referring to. Someone posted five steps for me to take and post answers to, and I did all five steps and posted the answers, albeit in two separate emails. What did I miss? I've submitted also a set of headers from a PUT request with an example of the line that goes missing when delivered to Tomcat (but not when delivered to Jetty). Until someone who knows more about Tomcat than I do (probably everyone on this list) confirms that the headers are actually being stripped, it's all probably moot. Chas. > c...@munat.com wrote: > ... >> >> So for Tomcat to unilaterally strip the entity-headers from a PUT >> request >> while passing only the optional entity-body seems just, well, wrong to >> me. >> >> For this reason, I'm guessing it's a bug. >> > Hi. > Just to situate things, I am not one of the Tomcat developers, merely a > user. So whatever I say about Tomcat below is relatively speculative and > under my own responsibility only. > Furthermore, my knowledge of REST is very scarce, but I gather that it > is a protocol/architecture (?) implemented, in this case, on top of HTTP. > I am a long-time user of HTTP, so in that respect, if only by virtue of > repetition, I feel on more solid ground. > > Tomcat is a servlet engine, and a HTTP server. It is not a REST server > nor client, and does not claim to be. > > This does not mean that you are wrong in your main contention; but means > that if Tomcat does things right with respect to the Java Servlet Spec > and the HTTP protocol, then Tomcat fulfills its contract, and whatever > would be missing to also make it be a REST server would not be a bug, > and merely an inconvenience. And if so, you /might/ be able to persuade > the developers of lifting the inconvenience, or better yet, submit a > patch for ditto. > > In your previous posts, you have not been very clear about what you > meant by "parameters", and later you have mentioned HTTP headers instead. > If Tomcat was indeed "stripping" HTTP headers from a PUT request (iow > making it impossible for a servlet to retrieve them), then I would tend > to agree that this is a bug, because I see nothing in the HTTP spec that > indicates that a PUT should not contain HTTP headers (at the contrary > even). > > But is that really what happens ? > Christopher asked you previously to confirm that point, but you didn't > really. > Or are you talking about parameters sent as part of the request line, > like in a query string ? > Can you give us an example of how you are sending these "parameters", > and how exactly you (attempt to) retrieve them in the web application ? > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [Fwd: Re: Parameters disappear from PUTs]
Are you serious?
I don't generate request headers. The browser does. And they do so
identically whether the HTTP method is PUT or POST and whether it's an
XMLHttpRequest or not. And I suspect that they've been doing so pretty
much since Marc Andreesen was still in Champaign-Urbana and Mosaic was
still an unrealized dream.
I'm certain you're not suggesting that browsers be forced to insert a name
before the parameter string in every POST request. Tomcat would instantly
cease to be a viable option for pretty much everyone. So why are you
harping on PUT?
In fact, what is it with this list? Is this the PUT Haters Club? Why do I
have to defend my decision to use PUT requests? Why do I have to keep
explaining an RFC that should be near and dear to the hearts of anyone who
works with web servers? What does *any* of this have to do with a simple
post to the list explaining that parameters passed with a PUT request seem
to be stripped out by Tomcat (though not by Jetty), and asking if this was
a bug or intended behavior?
It's a mystery to me.
Chas.
> On Fri, Feb 5, 2010 at 4:13 PM, wrote:
>
>> PUT /json/members/1b35d32f-714d-4393-b8c2-b4805e0c7a13 HTTP/1.1
>> Host: localhost:12344
>> User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US;
>> rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7
>> Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
>> Accept-Language: en-us,en;q=0.5
>> Accept-Encoding: gzip,deflate
>> Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
>> Keep-Alive: 300
>> Connection: keep-alive
>> X-Requested-With: XMLHttpRequest
>> Content-Type: application/x-www-form-urlencoded; charset=UTF-8
>> Referer: http://localhost:12344/
>> Content-Length: 297
>> Cookie: JSESSIONID=dexcmg3b1r45
>> emailAddress=bozo%40clown.com&title=Head%20Clown&nameGiven=Bozo&namesMiddle=The&nameFamily=Clown&namePreferred=Bozo&gender=Male&password=&passwordConfirm=&id=1b35d32f-714d-4393-b8c2-b4805e0c7a13&facebookName=bozo.the.clown&twitterName=i.am.bozo&flickrName=bozos.circus&linkedinName=mr.clown.to.you
>
> (RFC 2616)
> 4.2 Message Headers
>
> HTTP header fields, which include general-header (section 4.5),
> request-header (section 5.3), response-header (section 6.2), and
> entity-header (section 7.1) fields, follow the same generic format
> as that given in Section 3.1 of RFC 822 [9]. Each header field
> consists of a name followed by a colon (":") and the field value.
>
> Have you tried this using such a header format? e.g.
>
> Bozo-Fields:
> emailAddress=bozo%40clown.com&title=Head%20Clown&nameGiven=Bozo&yaddayaddayadda
>
> Just wonderin' ...
> --
> Hassan Schroeder hassan.schroe...@gmail.com
> twitter: @hassan
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>
-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Parameters disappear from PUTs
I believe that via a post on another list I have discovered the source of the trouble: According to the servlet specification... The following are the conditions that must be met before post form data will be populated to the parameter set: 1. The request is an HTTP or HTTPS request. 2. The HTTP method is POST. 3. The content type is application/x-www-form-urlencoded. 4. The servlet has made an initial call of any of the getParameter family of methods on the request object. So the failure is not Tomcat's, but the Servlet Spec's. There is no reason for this to be limited to POST -- both POST and PUT send entities to the server. I can only presume that this is a "feature" of Tomcat, ergo I won't bother with a bug report. Sigh. Chas. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [Fwd: Re: Parameters disappear from PUTs]
>> In fact, what is it with this list? Is this the PUT Haters Club? > > Mainly, it is your attitude. Given you are speaking to a community that > provides assistance to other members of the community for free, a less > argumentative tone would go a long way to help. Ah. I think I see where the problem lies. You think I came here to get something from you, and so you feel justified in making me jump through hoops to get it. But you have it backwards. I already had a workaround when I came to the list. And I'm pretty sure I'm going to abandon Tomcat anyway in favor of Jetty, or maybe something bigger. I posted to this list to help you. I was letting you know about a potential problem with Tomcat, and offering to help track down whether it was actually Tomcat that was causing that behavior or not. It seems counterproductive to me to attack people who come to help, but maybe that's just me. If I seem argumentative to you, consider that I've put in several hours on this trying to help and provide answers to people -- even when the questions are not germane. I've even backed everything up with evidence. And what I've gotten for my troubles is, frankly, nothing but grief. I am no further ahead than I was before I contacted the list, and I've spent time that I really can't afford to spend. > >> Why do I have to defend my decision to use PUT requests? > > When one of the possible solutions is a work-around and that work-around > may involve using a method other than PUT then I think it is fair for > folks to question the absolute need to use PUT. Workarounds are only solutions to people who care only for immediate relief, even if it costs them dearly down the road. A true solution doesn't require a workaround -- it fixes the problem. If you accept workarounds, then it won't be long before you find your code base an unmaintainable mess. I have a workaround. What I'm looking for is a solution, and using a POST for what should be a PUT is not a solution. At best, it's a hack. > >> Why do I have to keep >> explaining an RFC that should be near and dear to the hearts of anyone >> who >> works with web servers? > > Perhaps if you were using the correct terminology (entity body rather > than entity headers) folks here would find it easier to understand you. > Hassan was pointing out the correct way to use entity headers. You are > trying to use an entity body. If this was really Hassan's intent, then it might have been clearer if he said, "I think you are confusing the entity-body with the entity-headers. The paramaters that are being passed are actually the entity-body." I could live with that interpretation, although if you showed the parameters in the HTTP headers to virtually anyone I know, he or she would say "those are headers." So if you are right, it is actually at the expense of clear communication, rather than the reverse. But this brings up a good question. I used a Request Dumper Valve (at the suggestion of another person on this list) to check what Tomcat was getting. Although the parameters were definitely passed with the request, they were not present in the dump. The dump showed all the headers. Is the message body not included in the dump? If so, maybe that's where the parameters went. That would be an actual solution, and, in fact, it is where I believe RFC 2616 expects them to be. But note that the parameters are passed in *exactly* the same way in a POST and a PUT. When POSTed, there they are in the Dumper output. When PUTted, there they are not. If the RDV only dumps HTTP headers, and it dumps the parameters in a POST, then they are headers and should be in the PUT. If they are message body in the PUT, then so should they be in the POST, hence they should not be in the dump. Something is not consistent here. > >> What does *any* of this have to do with a simple >> post to the list explaining that parameters passed with a PUT request >> seem >> to be stripped out by Tomcat (though not by Jetty), and asking if this >> was >> a bug or intended behavior? > > I'll address that by replying to your other post that appears to have > identified the root cause. > > Mark > > > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Parameters disappear from PUTs
> I have just re-read section 9.6 of RFC2616. My understanding of PUT was > (and still is) that the entity body provided is used to create/replace > the entity at the provided URI. This is how Tomcat handles PUT requests > (it enabled) in the DefaultServlet. I agree that this is the intent of the RFC. But whether the entity is passed in the headers or in the body is really irrelevant. I see nothing in the RFC to say that the entity *cannot* be passed via the header. As I mentioned in my other email, the parameters are sent by the browser identically for POSTs and PUTs. Nothing in the RFC leads me to believe that it intends for POST entities to travel in the header, so this is probably just a convention established by the browsers. But again, nothing in the RFC leads me to believe that this behavior is prohibited. And even if it is, that's a moot point. Just getting browsers to do PUTs is a real trick (I use AJAX). Tomcat is either going to permit the context to get at those parameters, or it isn't. > > My understanding of your PUT request is that the entity body is a series > of parameters that are used to create an entity that is then made > available at the provided URI. That doesn't match exactly with what I > expect having read RFC2616. It matters not at all what the entity is that is to be stored, or how it is stored. All that matters is that the server stores it at the specified URI in such a way that it is recoverable via a GET. I PUT; I GET. The entity itself could be a JPEG, an MP3 file, a binary program, an anonymous function, a properties list, a text file, HTML, a style sheet, or plain gibberish -- as long as it gets associated with that URI and comes back the same as it went in (barring other PUTs or DELETEs). How does your understanding differ from this? Example: I could do a PUT of an XML file containing Hamlet to: http://mysite.com/plays/Hamlet Then I could retrieve a part of that play via: http://mysite.com/plays/Hamlet/scenes/I I might then correct some problems, and do a PUT of that scene back to the above URI. Later, I might to a GET to http://mysite.com/plays/Hamlet and get back the whole play with the revised Scene I. The entity is a black box. It may contain sub-entities, each of which is its own black box. HTTP should not care. I PUT and entity, I GET an entity. It's actually very simple > > There is some wiggle room in the specification. SRV.3.1.1 states that: > > The following are the conditions that must be met before post form data > will be populated to the parameter set: ... > > > A strict reading of SRV.3.1.1 only refers to how to handle POST data. It > makes no reference to PUT data. Therefore, it would not (in my view) be > a breach of the spec to treat application/x-www-form-urlencoded PUT > requests in the same way as application/x-www-form-urlencoded POST > requests. This should probably be optional since this is a grey area. I > would be in favour of an enhancement request for Tomcat 7 that > implemented such a feature. Yes, I noted that same thing: these are conditions that "must be met before *post* form data will be populated to the parameter set." I believe that the authors weren't even thinking of PUTs at the time (it was 1999, right?), so I don't think this was meant to apply to PUTs, and I think the standard doesn't really address them. The folks at Jetty seem to agree, since AFAICT, Jetty passes the parameters along. But getting the Tomcat folks to change this is probably an uphill battle, and I can just switch to Jetty. I think it would be a very good idea for Tomcat to do it, though, so if someone here has that kind of pull, by all means... And that's all the time I have for this. Good luck with it, and thanks for your replies. Chas. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: Parameters disappear from PUTs
> Just to to clarify this point, the entity in the example wireshark trace > you provided is in the request body, not the headers. I wasn't aware that I had provided a Wireshark trace, but that's good to know. (Wireshark would not install without a fight on my Mac, so my brother ran the trace on his and just reported the results to me.) I haven't used Wireshark since it was Ethereal. On the other hand, I examined the "headers" outgoing from Firefox using Live HTTP Headers tool and the parameters were there, whereas when I use the tool to look at GET request headers, the body of the request is not. So perhaps that is the source of the confusion over whether they are in the headers or not. Live HTTP "Headers" appears to be a misnomer. >> And even if it is, that's a moot point. Just getting browsers to do PUTs >> is a real trick (I use AJAX). Tomcat is either going to permit the >> context >> to get at those parameters, or it isn't. > > Again, using the wireshark trace you provided as an example, Tomcat will > allow access to the request body via ServletRequest.getInputStream(). If > this were a POST request, Tomcat would read the input stream, convert it > to parameters and make those parameters available via > ServletRequest.getParameters(). I assume that this is what Jetty is > doing for both PUT and POST. So if I read this correctly, the PUT body containing the parameters should be available via ServletRequest.getInputStream() despite them not being included in the Dumper Valve output? If that's the case, then I can work with that and still use a PUT request. I still think that there's no reason that Tomcat can't parse the parameters and serve them up same as for a POST, but at least they're not being stripped out! >> that the authors weren't even thinking of PUTs at the time (it was 1999, >> right?) > > Before my time, but that text hasn't changed even in the most recent > version of the spec from Dec 2009. Maybe because no one asked them to. I'm pretty sure that PUT is still the bastard step-child of HTTP. >> , so I don't think this was meant to apply to PUTs, and I think the >> standard doesn't really address them. The folks at Jetty seem to agree, >> since AFAICT, Jetty passes the parameters along. > > It is a completely grey area. Both Tomcat and Jetty are fully compliant > with the Servlet spec and the HTTP spec. At the moment Jetty converts > the request body to parameters and Tomcat doesn't. No data is lost, you > just need to use a different approach to access it with the different > servers. That was the essential question of my initial post to this list. Thank you very much for clarifying. >> But getting the Tomcat folks to change this is probably an uphill >> battle, > > I'm guessing you didn't realise I was one of the Tomcat committers. As I > said previously, I'm not against making a change in this area. I had no idea. I'm glad to be talking to someone who can actually make a decision. That makes a lot of previously irrelevant discussion relevant. I'm happy to discuss and consider different views on the issue if it might actually lead to something and isn't just gratuitous argument. >> and I can just switch to Jetty. I think it would be a very good idea for >> Tomcat to do it, though, so if someone here has that kind of pull, by >> all >> means... > > I'll add an enhancement request to Tomcat 7. Because this is a grey area > other folks using Tomcat may be expecting the data in the request body > and changing that would break things for them. It will certainly have to > be optional. As to which should be the default (convert or not convert) > I'm still on the fence. Implementing this isn't going to be at the top > of my todo list (that is still the Servlet 3.0 work) but at least if it > is in Bugzilla it won't get forgotten about. Why optional? Why doesn't it just do both? Isn't it possible to parse the parameters (if they are recognized as such) and make them available via the same mechanism as POST parameters, but to also leave a copy in the message body for those who prefer to retrieve them there? It seems to me that that would require no changes to configuration (always a pain) and no breaking changes to the API. Am I missing something? I really appreciate your professionalism and your thoughtful response to my email despite my obvious frustration. I am glad we were able to resolve the issue to the benefit of all, and I apologize for my earlier testiness. I should not have rushed to judgment before all the options were exhausted. Chas. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [Fwd: Re: Parameters disappear from PUTs]
epeatedly, change nothing but the method from PUT to POST and Tomcat handles it fine. So either Tomcat is handling bad requests or the request is actually properly formatted. Regardless, it is irrelevant to the question at had, which is whether application/x-www-form-urlencoded form data should be made available whether the request method is GET, POST, or PUT. I am saying that this is a legitimate use of PUT and there is nothing in any of the RFCs or the Servlet Spec that I can find that prohibits this behavior and that at least one other servlet container implements it, so why not Tomcat? The rest of this brouhaha is just a gigantic waste of my time rehashing arguments that have already been made and, to my mind, refuted, and wasting the list's bandwidth. >> I can find no justification in RFC 2616 for stripping >> this line out -- it is part of the entity-headers as mentioned above, >> and >> the RFC also discusses transparency, stating that headers not included >> in >> the standard list should be passed through. On a POST request, which is >> virtually identical to the above, they appear in the request dump. > > As headers? Or as request parameters? Take your pick. Chas. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [Fwd: Re: Parameters disappear from PUTs]
I posted it exactly as it came to me. > On Saturday 06 February 2010 03:27:23 c...@munat.com wrote: >> Are you serious? > > [...] > >> I'm certain you're not suggesting that browsers be forced to insert a >> name >> before the parameter string in every POST request. > [...] >> What does *any* of this have to do with a simple >> post to the list explaining that parameters passed with a PUT request >> seem >> to be stripped out by Tomcat > [...] >> It's a mystery to me. > > If you had properly formatted your message in a way that request headers, > request body and response could be distinguished, you would've gotten a > useful > response sooner. I also misread this several times before I noticed, > because > it all looked like a big block of headers (quote characters intentionally > removed): > > PUT /json/members/1b35d32f-714d-4393-b8c2-b4805e0c7a13 HTTP/1.1 > Host: localhost:12344 > User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; > rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 > Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 > Accept-Language: en-us,en;q=0.5 > Accept-Encoding: gzip,deflate > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 > Keep-Alive: 300 > Connection: keep-alive > X-Requested-With: XMLHttpRequest > Content-Type: application/x-www-form-urlencoded; charset=UTF-8 > Referer: http://localhost:12344/ > Content-Length: 297 > Cookie: JSESSIONID=dexcmg3b1r45 > emailAddress=bozo%40clown.com&title=Head%20Clown&nameGiven=Bozo&namesMiddle=The&nameFamily=Clown&namePreferred=Bozo&gender=Male&password=&passwordConfirm=&id=1b35d32f-714d-4393- > b8c2- > b4805e0c7a13&facebookName=bozo.the.clown&twitterName=i.am.bozo&flickrName=bozos.circus&linkedinName=mr.clown.to.you > HTTP/1.x 201 Created > Content-Length: 82 > Content-Type: text/plain; charset=UTF-8 > X-Lift-Version: 1.1-M4 > Server: Jetty(6.1.22) > >> Chas. > > Rainer > > - > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > > - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Re: [Fwd: Re: Parameters disappear from PUTs]
, Tomcat could buffer this >and still make the data available as an InputStream to the servlet >as well as request parameters, but this requires additional >processing on Tomcat's behalf as well as memory and/or disk space >to buffer that data. This seems like an architectural issue specific to Tomcat. I can't speak to that as I know nothing about how Tomcat is implemented. If it's too difficult or requires a significant performance hit to do both, then it's probably not worth it. > 2. If Tomcat must consume the input stream, it means that Tomcat must >read everything before the user's code can continue. This >prohibits streamed processing of input data. If the input data is a >multi-megabyte XML document intended for SAX processing, Tomcat's >"interference" in this process is a complete waste of time and may >result in unnecessary OOMEs or disk writes and reads. >I would agree that a client sending XML data as www-form-urlencoded >content type would be pretty stupid, though. I agree on all counts. > 3. It violates the servlet specification. This may sound like a >cop-out, but there is a good reason that specifications exist: >so that users can experience expected behavior under certain >circumstances. In this case, the specification does not /prohibit/ >the parsing of www-form-urlencoded data in a PUT request, but it also >does not say that it /will/ be done. From my perspective, Jetty's >handling of this situation is "surprising". I'm not clear on how it violates the servlet spec. I agree that following specs is best practice and I'm not at all a fan of folks inventing their own ways of doing things (I still have a contract out on Marc Andreesen, if anyone knows where he is). Ten years of cross-browser hell made sure of that. But if you're only parsing the body as an option, and leaving it there (having your cake and eating it to, so to speak), I don't see how that violates the spec or the RFC. Yes, stripping out the body = bad, bad, bad. But params parsed for PUTs, *in addition to the body*. How is that a violation? > Aah, we've wasted more breath on other issues as well. Come back to us > when logging or character encoding rub you the wrong way. Then, we'll > really waste all our breath. :) Sweet Jesus! Do NOT get me started on character encoding. (Actually, the encoding is not so bad. It's the obscenely twisted implementation of it in most places that drives me batty.) >>> As headers? Or as request parameters? >> >> Take your pick. > > There is a big difference between these two, and no, it's not > smoke-and-mirrors. The HTTP specification defines each quite > specifically, names them, describes their use, and shows how they are > ultimately laid down in the actual messages that traverse the network. Actually, my point wasn't that it made no difference to Tomcat, but that it made no difference to me how the form data got to me via the PUT. Headers, footers, armpits -- just show me the form data. My concern was that it didn't appear in the Dumper Valve output *at all* (granted, I am taking my brother's word on this). If it's there and I missed it, then this has all been a fun diversion. Ha, ha. Nothing to see here. Move along now. > One of the sources of confusion at the beginning of this thread was that > you were calling this data "params in the header" which muddled these > terms together. Why do you keep torturing me? OK! OK! No dessert for me tonight. Satisfied? > I'm glad we were able to identify the problem, if not resolve it quite > to your satisfaction. Actually, if the data is there in the body and available, I'm happy. It may not appear so at first, but I am actually smart enough to write a parser for it myself! (In Scala it's like one line of code.) Thanks a lot for your replies. They've been very educational and have caused my paranoia to wane considerably. There are some pretty bright people on this list. Chas. - To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
Windows service for tomcat7
Hi, I'm trying to install tomcat service for Windows 7 manually on my system through command prompt. I tried with">>service.bat install MyProject" but it looks like service doesn't get installed properly because I can see the service in the list but when I start it, it throws error. Any idea on what could be the issue? I'm not using tomcat's installer for windows as I would like to bundle it along with my project installer with bundled Java as well. I could use other Windows batch scripts of starting and shutting down tomcat which work well. Thanks
