Also to note, I'm unsure if the TZ in the header is being sent or if there would be a change in those. The F5 offloads connections to the box, so every session is a local SNAT from the F5, but it's something to investigate. Thanks again for your leads everyone!
On Fri, Oct 28, 2022, 09:22 David <rax1...@gmail.com> wrote: > I'll see if I can answer all queries. > > The server doesn't migrate, at least not across timezones. It is a > nutanix virtual though, so migration between hosts locally is possible. > > There is a Java app that allows a user to reset their password. The db > is called to retrieve a generated code for them to use to reset, it also > validates the user hasn't done this more than 3 times since having a valid > login, and sets an expiry for the duration of this code (4hrs). > > There is a dbutil Java app that does use SimpleDateFormat, so I will look > into the bug provided. > > When the switch happens, it will show that the date being sent to the DB > by the Java reset pw app is sending values in UTC. It is instant and > precisely on time, since it's +5 hours, and the pw reset is only valid for > 4, it breaks this functionality. The Java app lives in tomcat and is a > subset of the main website app. > > We have lower environments where this doesn't occur. The app code is the > same everywhere, Java and OS times are stable everywhere as well. > > I did notice that stuck valve detection had a lot of hits during one > occurrence, tomcat slowed as connection threads climbed, and the switch to > utc being sent to the db happened at this time. I was thinking it may be a > db2 contention issue based on this, normally that would cause connections > to back up and tomcat to run out of threads though, I've never seen it > change cdt to udt. I'll dig more into the SimpleDateFormat, as it may be > the cause but is triggered by load/db contention. Lower environments that > don't show the issue don't have much traffic as they are for dev/uat/etc. > > Thanks all! > David > > On Fri, Oct 28, 2022, 02:51 Konstantin Kolinko <knst.koli...@gmail.com> > wrote: > >> чт, 27 окт. 2022 г. в 18:31, David <rax1...@gmail.com>: >> > >> > Hi all, >> > >> > I've experienced an issue since the morning of the 21st that I'm >> > hoping to get some direction on for where to look. >> > >> > An app uses the date/time to set a timeout for a password reset. >> > This had been working fine for years and suddenly it failed. A restart >> of >> > tomcat allowed it to work for a day, then 12 hours, then 5 hours, then >> 1 hr >> > and now is averaging a 10 minute or so working duration between tomcat >> > restarts. >> > >> > Changing the logging in the app showed that the issue is due to it >> > sending UTC to the DB while it is broken. Restarting Tomcat results in >> CDT >> > being sent for a while until randomly it switches again. >> > >> > RHEL 7.9, jvm 1.8.0_231-b11, Tomcat 9.0.29 >> >> Looking at the changelog since 9.0.29 onwards, >> there was the following issue, fixed in 9.0.34 two years ago: >> >> https://bz.apache.org/bugzilla/show_bug.cgi?id=64226 >> "Tomcat 9 can return HTTP date headers in timezone other than GMT" >> >> Fixed by commit >> >> https://github.com/apache/tomcat/commit/f64a2a7150bff01bca479c7c319b7e8db879df26 >> >> It is about how parsing a date time string may affect formatting time >> if the same SimpleDateFormat instance is reused. It is triggered by a >> client sending a header using a different time zone. >> >> >> It is unlikely your issue with the database (unless your application >> uses Tomcat internal classes such as ConcurrentDateFormat or >> FastHttpDateFormat), but it affects HTTP headers generated by Tomcat. >> >> Best regards, >> Konstantin Kolinko >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org >> For additional commands, e-mail: users-h...@tomcat.apache.org >> >>
