Daniel,
On 8/13/25 8:59 PM, Daniel Schwartz wrote:
Chris,
You have given me a lot to think about.
Re your comment: "Getting ApacheBench is different depending upon your development
environment. If you let me know your OS, etc. I can point you in the right
direction."
I found the following via Google:
https://httpd.apache.org/docs/2.4/programs/ab.html
This indicates that it comes as a part of the Apache HTTP server and is useful
only for this. Is there a version that works with Glassfish? Is there any
documentation?
ab is a utility that you can run against any web server, including both
Glassfish and Tomcat. It *does* come with Apache httpd, but you can get
it separately and don't have to install the web server to use it.
I'm running on Windows on AWS.
Apache Lounge [1] has binary versions of Apache httpd which include the
utility. I didn't see any option to just get ab.exe from there, but you
can download the ZIP package for Apache httpd and just take what you need.
ChatGPT tells me that you will need these files from the ZIP archive:
ab.exe
Required DLLs (usually):
libhttpd.dll
libapr-1.dll
libaprutil-1.dll
pcre.dll or pcre2-*.dll (depending on version)
ssleay32.dll, libeay32.dll, or OpenSSL DLLs (if SSL support is needed)
Just extract all those files into the same directory and run "ab" from
the command line.
It's a CLI tool, so you'll do a lot of typing. I would refer to the
documentation[2] to see what all the options are and what they mean.
I would start with something simple like this:
C:> ab -n 10 https://example.com/yourapp/page
This will make 10 requests and give you some statistics. Then check your
Glassfish pool counters.
If you get a weird error like "apr_socket_connect(): Invalid argument
(22)" or something, try using "127.0.0.1" instead of "localhost".
-chris
[1] https://apachelounge.com/downloads
[2] (https://apachelounge.com/downloads)
-----Original Message-----
From: Christopher Schultz <ch...@christopherschultz.net>
Sent: Wednesday, August 13, 2025 2:49 PM
To: users@tomcat.apache.org
Subject: Re: [EXTERNAL EMAIL] RE: How to access a REST service
Dan,
On 8/12/25 11:33 PM, Daniel Schwartz wrote:
-----Original Message-----
From: Robert Turner <rtur...@e-djuster.ca.INVALID>
Sent: Tuesday, August 12, 2025 11:09 PM
To: Tomcat Users List <users@tomcat.apache.org>
Subject: Re: [EXTERNAL EMAIL] RE: How to access a REST service
> [snip]>
For clarity, the 13 ms is the time it takes the pool to return to the caller
(Java code) a connection when requested (as opposed to how long the connection
is used for). Your other comments are accurate -- you could have many idle
connections, however, 300 seconds is fairly reasonable for an idle timeout.
What you want to be looking for is why you are peeking so high (maybe a benign
connection that does nothing is triggering DB connections -- if so, this could
lead to a DoS of your server with minimal effort). Something is either holding
on to connections for a long period, or you are getting lots of connections at
once. You should be able to tell easily which one it is because you will have
access logs, and you can annotate your code with diagnostics as needed. Also,
you can simply test on your development computer and see what's going
on...without exposing your testing to the wide world of the Internet and the
associated risks.
DGS: Yes, I have access logs. I will look more closely and see if anything
stands out. Within nginx you can block requests from specific IP addresses,
but I haven't resorted to this yet.
DGS: Am not sure what you mean by "you can annotate your code with diagnostics as needed"
or "you can simply test on your development computer and see what's going on". What sort
of diagnostics, and what sort of test?
I suspect what Robert means by "annotate your code with diagnostics as needed" just
means "put something in your code to record additional information. For example, logging
the request URI, number of db connections obtained, the amount of time each connection took to
obtain (~13ms ain't bad, but it's not exactly lightning-fast, either), and *the amount of time
you spend doing your queries before the connections are returned to the pool* which seems to
be something worth knowing.
As for "what sort of test", I would recommend setting up a load-test in a development
environment like your own workstation. Get your application running locally with the smallest pool
you can muster (8 connections? Weird that there is a minimum number of connections in this pool...
I'm not impressed). Make a few requests manually to ensure that the application is working
"normally". Then, unleash a load test on the application.
There are a number of ways to do this.
First, you can mash RELOAD in your web browser and see what happens.
That's usually not very helpful and doesn't give you much useful data.
Another simple load-test is to use curl from the command line like this:
$ while true ; do curl
https://urldefense.proofpoint.com/v2/url?u=http-3A__localhost-3A8080_your_application_page&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=CTy2h5PJHscrNmr4jn0Y-7PvWGT5hMBTsJ5_E5Vs4YvyNG7N7W-VPGs88HyDGBUV&s=di3ilaaf964xSIwNBTXtkVqVtJr5iPLPVNWhYYVb9Xw&e=
> /dev/null; done
That will pound your application harder than you probably can through your web
browser. Pretty CTRL-C to stop the madness.
When you are ready for proper load-testing, consider a simple tool like ab [1].
Getting ApacheBench is different depending upon your development environment.
If you let me know your OS, etc. I can point you in the right direction.
When you are ready to go down a very deep rabbit hole, you can use JMeter[2]
but I suspect you will be able to learn a lot using both curl and ab, with ab
being the much better choice.
I'm hoping you are able to get some questions answered about your application
like:
1. If I configure the pool with 8 connections, how many requests (in
series) are required to get the pool to break?
2. If I configure the pool with 8 connections, how many requests (in
parallel) are required to get the pool to break?
3. If I configure the pool with 100 connections, how many requests (in
series) are required to get the pool to break?
4. If I configure the pool with 100 connections, how many requests (in parallel
are required to get the pool to break?
5. What's the average response-time for a large number of serial requests? Hopefully, it's "fast".
For most users, a second of latency is about when people start getting bored waiting for your site and
consider switching to a competitor. Most users are also willing to wait a lot longer for some specific
things. But loading your home page isn't one of them. "Generate a complex report for me?" Sure,
they'll wait. But "click on the item to get details" is not something you want to make people wait
for.
6. What's the average response-time for a large number of parallel requests,
mixed-in with serial requests at the same time. This simulates *real world
traffic*. And your application needs to be able to meet
*peak-load* at any given time. So if you expect 100 simultaneous requests, your
application must be able to handle those requests without falling over. I can
tell you that if you have a fairly well-written application, running on Tomcat,
with a single connection in the database connection pool, and if your
application can respond to requests in 1 second, and you have a 60 second
wait-for-connection configuration on that pool, then your application can
handle a burst of traffic up to 60 simultaneous requests without rejecting a
single one. Once the 61st request is made (on top of the other 60 simultaneous
requests), one of them is going to throw an error. But the application will not
crash.
Tomcat will not crash. The JVM will not crash. A single error for a single
request will be returned to a single client. All 60 others will get the
expected response. If you crank that up to 100 simultaneous requests, over and
over again, 60% of them will return successfully and the other 40% will get
errors. Which is exactly what you want. If your database (which is the real
bottleneck, here) can really only take
(e.g.) 60 simultaneous requests, then don't configure your application to give
it 100 or 1000. It's better to return errors to some clients than to slowly
grind everything to a halt. If your database starts thrashing, then you have
nothing.
The "in series" questions are easy to answer with curl, and hopefully the answers are
"I can make a large number of requests in series and never break the pool" regardless of
the configuration.
The "in parallel" questions will get down to what happens when the pool is
empty, how long are your db connections being used (i.e. duration between pool-check-out
and pool-return), etc., and will require that you use a tool like ab or JMeter. ab is was
easier, so, again, I'd recommend it for your uses.
ab gives you great tabular data when it terminates, and will answer basically
all the questions above if you run it with various options.
Make sure to keep an eye on the application server's log files. For Tomcat, that means
catalina.out plus any application-log file you may have configured in your application.
You are looking for any errors and especially stack-traces. For the "simple
tests" (like you at the browser or trying 100 requests back-to-back) you should see
no errors.
But you should start seeing errors at some point. And the server (Glassfish, or
Tomcat) should *not* crash. Both the JVM and the server should stay running,
and so should your application. Your application and configuration *do* have a
breaking point: that's the point of a
load-test: to understand what that breaking point actually is. But you
shouldn't be able to take-down the application server or the JVM.
Hope that helps,
-chris
[1]
https://urldefense.proofpoint.com/v2/url?u=https-3A__httpd.apache.org_docs_2.4_programs_ab.html&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=CTy2h5PJHscrNmr4jn0Y-7PvWGT5hMBTsJ5_E5Vs4YvyNG7N7W-VPGs88HyDGBUV&s=JCzN9UqN9UNmQkeTGL_eK59RndvTL3wnS3hkMKeqR8s&e=
[2]
https://urldefense.proofpoint.com/v2/url?u=https-3A__jmeter.apache.org_&d=DwICaQ&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=AbCalLxzopgQUG9LLcXdB80OM-GtDfItX76RMxNYqz4&m=CTy2h5PJHscrNmr4jn0Y-7PvWGT5hMBTsJ5_E5Vs4YvyNG7N7W-VPGs88HyDGBUV&s=AzXpplYNVaVb3NtWqdjjZqsfT36_PTaOGOe6-CpbpBo&e=
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org
