-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Jerry,
On 8/15/18 4:00 PM, Jerry Malcolm wrote:
> On 8/15/2018 1:50 PM, Olaf Kock wrote:
>> Jerry,
>>
>>
>> On 15.08.2018 18:14, Jerry Malcolm wrote:
>>> I have a mobile app that issues several http web service calls
>>> to initialize. I was making them sequentially with no issues.
>>> I then changed to give them all separate threads so they could
>>> load asynchronously. Then the bottom fell out. I started
>>> getting empty responses and some responses with results of
>>> three or four of the calls concatenated. I traced the problem
>>> from the app back through apache through mod_jk and found the
>>> culprit to be Tomcat.
>>>
>>> I'm a seasoned Tomcat developer for over 15 years. I've never
>>> seen anything like this. But it's really scary. What I found
>>> is that sometime during the execution of one servlet call, it's
>>> thread data is swapped to thread data of another servlet call.
>>> I know this sounds like Twilight Zone. But here is a log
>>> output. At the beginning of doGet(), I generated a random text
>>> string just to keep track of the thread data/:/
>>>
>>> Thread: ajp-nio-8009-exec-24 uid: rclebgb --> Thread:
>>> ajp-nio-8009-exec-29 uid: ceycfqd --> Thread:
>>> ajp-nio-8009-exec-29 uid: ceycfqd <-- Thread:
>>> ajp-nio-8009-exec-24 uid: ceycfqd <--
>>>
>>> Note that when thread 24 starts I store the "rcl..." uid.
>>> Another call comes in and starts thread 29. By the time thread
>>> 24's servlet code is done, it now has thread 29's data. (The
>>> uid is just a quick variable for reference. The request
>>> object, response object, EVERYTHING is now thread 29's data).
>>>
>>> This explains why I'm getting empty responses and other
>>> response with the data for multiple requests concatenated
>>> together. The "rcl..." instance data has totally disappeared,
>>> and all of the server calls are now using the "cey..." instance
>>> data (i.e. response object).
>>>
>>> I figure this is some sort of timing/race condition that only
>>> occurs with a second call coming in while the first call is
>>> still in progress. I can go back to sending the mobile app
>>> calls serially and probably work around this. But this is a
>>> huge problem.
>>>
>>> Any ideas? (BTW... Tomcat 9.0.7)
>>>
>>
>> As we don't know which code generates this log output, it's hard
>> to judge what actually causes your problem. You say "thread data"
>> is being swapped, and the very first aspect that comes to my mind
>> is: Servlets are inherently multithreaded, and a common pattern
>> of bugs is if a servlet has a member variable that is used for
>> request processing: There typically is only one Servlet object
>> ever, thus they all share the same state, if the state is stored
>> in a member variable. This might be directly in the servlet or in
>> some other component or singleton somewhere.
>>
>> Any state and request processing must be done on the
>> request/response pair, and properly threadsafe in every other
>> part of your code.
>>
>> And most likely this is an issue that luckily shows up when
>> you're issuing a lot of parallel threads due to parallelizing one
>> client. It'd be a lot harder to reproduce if it were individual
>> users, who (very) occasionally see the wrong data. Consider
>> yourself lucky to have such a nice and reproducible issue.
>>
>> Olaf
>
> Olaf,
>
> I'm having a bit of trouble feeling "lucky" that I have a problem
> that is catastrophically blocking my development progress toward a
> deadline.....
Uh.... <adjusts collar nervously>.... hmm.
> I'm not sure what you mean by typically there is only one servlet
> object. There's one class. But a new instance is created on each
> request, right? And all instance variables should be scoped to
> that instance alone, right?
Nope. I think that's where everything you're observing is coming from,
then.
> It's fundamental to Java that a private local variable can't
> arbitrarily change to another value in the middle of executing a
> method other than by the local code changing it itself, right?
Since servlet instances are shared, so are all those private variable
references.
> Yet uid (and all the other variables including request and
> response) changed.. somewhere between the beginning and ending of
> doGet(). !! I'm not doing any singleton stuff. Simply
> request-->Servlet-->response.
So, that's a scary thing or me to hear: "and all the other variables
including request and response". Are you asking about the parameters
to the e.g. doGet() method, or are you saying that you also have
"private HttpServletRequest request" defined in your servlet and you
are somewhere doing this:
public void doGet(HttpServletRequest request, HttpServletResponse
response) {
this.request = request;
this.response = response;
... other stuff
}
?
> My servlet code is very basic. It is not multithreaded.
>
> public class MyServlet(....) { private String uid; // <====
> PRIVATE access!!
Yes, private. But only one instance of MyServlet exists in memory and
it will be called by many threads, each calling doGet().
Now, the objects passed-into that method will be thread-specific, at
least from your perspective.
> doGet() { uid = createUID(); System.out.println( "Thread: " +
> Thread.currentThread().getName() + " uid: " + uid + " -->");
>
> <do processing>.... (no new threads, totally synchronous, simply
> get a couple of vars from a JDBC call)
>
> System.out.println( "Thread: " + Thread.currentThread().getName() +
> " uid: " + uid + " <--"); }
If you try this, you might get better results, or at least less-scary.
doGet() {
String uid = createUID(); // This has local-scope, now
System.out.println( "Thread: " + Thread.currentThread().getName()
+ " uid: " + uid + " -->");
<do processing>.... (no new threads, totally synchronous, simply
get a couple of vars from a JDBC call)
System.out.println( "Thread: " + Thread.currentThread().getName()
+ " uid: " + uid + " <--");
}
> That's it. I can think of nothing I could be doing that would
> cause the PRIVATE uid varible to be different on the 2nd println
> from the 1st println. And furthermore, the 2nd println has a uid
> from a DIFFERENT request.
>
> What could I possibly be doing in this code above that could cause
> this?
Simply not realizing that the servlet instance is shared is your
problem, here. From what you've shown, only the "uid" field is
horrible non-thread-safe. What about the rest of the code in the servlet
?
If you are storing request and response references in a similar way,
then your servlet is fatally flawed. If you aren't doing that kind of
thing, then maybe your application is fine... just this one
ill-conceived test has scared you into investigating.
- -chris
-----BEGIN PGP SIGNATURE-----
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/
iQIzBAEBCAAdFiEEMmKgYcQvxMe7tcJcHPApP6U8pFgFAlt0iesACgkQHPApP6U8
pFhApw/+PMB8S8ni287N3ONjnzXX6pkNNj5TclQvO+wFANPyghArrvFQymzGTnpd
e3sslgyQcyrmP5Vxf6z9AQp8Gp2nCTVYhzx+h4mBqFv/4bboOx201Gcfw7JT3LNR
UGndfnrk6M6lb5dVKnXhFxSgmfc4Hia4gdgV8vPR25BHgc6XM/C0xstxU0V+Ft3D
qz1edo2zPJikLwSuECCoHF/aESwOLS4acCHAoQlPFhqXtnlwh0og9HsvMtgvfXvi
ax1qKOQ73VRn+WLl32Co+xSHO9sw70Nxp92sO5Ku1HCFWe8xfzrdEbKEGT/1w74S
hf5sMAF5ek2UU3CcDaTk+hO/fMRNLiyFKCAeXqiZTsA43/su3ScPYpXQixiXdDq0
yW/oOT+jc3DKuW/0qnhTSCQ+CS4nRh6UAwQKlrMCwv4kkLsKHm69d5VsJ03MWifB
roSfTBp1FrxSZG0VqMoaLXZQBiBU8YAniRLzI5c+xP8lf9I8rYPta1RdVMS8aAq3
jMHDB2Z65S9gSp7PkODFSKe7w44EItIbUHbKTyHqSVX9GWSfRy1NDRFZigcjin61
q/3xs4Pz5924+zsAjCiWpVKE/IS3LOsrbgrW80f6vAlrfHzJMQIJW4RJ3SuxqAds
9qj6Eywq/syq74NLSpKZMMsDdPxg12vyBhczmguPGtTdNNbUAMg=
=XfhV
-----END PGP SIGNATURE-----
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
