Quoting Brett Palmer <[email protected]>:
*Adrian,
I ported our code over to the newest ofbiz trunk and tried out the changes
for the new service engine. The changes that you made are working very
well. I configured our server to use multiple pools and then scheduled
various jobs to those pools. Servers that were not configured to service
the pools left the jobs dormant and servers that were configured to service
the new pools did so as expected.
To be clear, that has always been the behavior of the Job Scheduler. I
did not change that.
These new changes will work for us in our production environment. Thanks
for implementing them. I now need to work on tuning the configuration
settings based on the particular jobs and the capacity of our servers.
The changes were intended to prevent the Job Scheduler from saturating
the server under heavy load. So, those are the results I would be
interested in hearing about.
I did have another question on the thread-pool configuration. Here is a
sample configuration that I was using to do some of the testing.
<thread-pool send-to-pool="pool"
purge-job-days="4"
failed-retry-min="3"
ttl="120000"
jobs="100"
min-threads="2"
max-threads="5"
wait-millis="1000"
poll-enabled="true"
poll-db-millis="30000">
<run-from-pool name="pool"/>
<run-from-pool name="testPool"/>
<run-from-pool name="pool2"/>
<run-from-pool name="pool3"/>
<run-from-pool name="pool4"/>
</thread-pool>
Is the “send-to-pool” attribute the default pool that is used by the
service engine for any sync and async requests through the service engine
API?
Correct.
Is there a relationship between the “send-to-pool” attribute and the
“run-from-pool” names or are they independent of each other? For example,
if I don't have a run-from-pool element with a name="pool" will the default
"pool" still work?
They are not related. If you don't have a
<run-from-pool name="pool"/>
element somewhere, then jobs sent to the "pool" pool will not be serviced.
Thanks again for you work on the service engine. We really appreciate it.
Let me know if you need more feedback on the new changes.
Brett*
On Mon, Sep 17, 2012 at 3:50 PM, Adrian Crum <
[email protected]> wrote:
Brett,
Any news on this?
-Adrian
On 8/18/2012 4:11 PM, Brett Palmer wrote:
Adrian,
I need to update our application to work with the newest code from the
trunk. We did an update a couple months ago so I don't think it will take
me long. I'll try to provide some feedback early next week.
Brett
On Sat, Aug 18, 2012 at 12:31 AM, Adrian Crum <
adrian.crum@sandglass-**software.com <[email protected]>>
wrote:
Brett,
Any results yet?
-Adrian
On 8/8/2012 12:57 AM, Brett Palmer wrote:
Adrian,
Thanks I'll take an update and try it out.
Brett
On Aug 7, 2012 4:23 PM, "Adrian Crum" <adrian.crum@sandglass-**
software.com
<adrian.crum@sandglass-**software.com<[email protected]>
>>
wrote:
Brett,
I think I solved your problems with my recent commits, ending with rev
1370566. Let me know if it helps.
-Adrian
On 8/5/2012 4:53 PM, Brett Palmer wrote:
Adrian,
Thanks for the update. Here are some feedback points on your listed
items:
1. JobPoller get out-of-memor error. We've seen this a lot in
production
servers when the JobSandbox table is not constantly pruned of old
records.
It would be nice if the poller restricted its search for only
active
records it could process.
2. Queue for capturing missing records would be good. From item 1
above
we
have had locks on table when the poller is busy doing a scan and new
jobs
cannot be added or time out.
Other wish items:
- Ability to assign different service engines to process specific job
types. We often multiple application servers but want to limit how
many
concurrent jobs are run. For example, if I had 4 app servers
connected
to
the same DB I may only want one app server to service particular jobs.
I
thought this feature was possible but when I tried to implement it by
changing some of the configuration files it never worked correctly.
- JMS support for the service engine. It would be nice if there was a
JMS
interface for those that want to use JMS as their queuing mechanism
for
jobs.
Brett
On Sun, Aug 5, 2012 at 6:21 AM, Adrian Crum <
adrian.crum@sandglass-****softwa**re.com <http://software.com> <
adrian.crum@sandglass-**softwa**re.com <http://software.com><
adrian.crum@sandglass-**software.com<[email protected]>
>
wrote:
On 8/5/2012 11:02 AM, Adrian Crum wrote:
I just committed a bunch of changes to the Job Manager group of
classes.
The changes help simplify the code and hopefully make the Job
Manager
more
robust. On the other hand, I might have broken something. ;) I will
monitor
the mailing list for problems.
I believe the JobPoller settings in serviceengine.xml (the
<thread-pool>
element) should be changed. I think min-threads should be set to "2"
and
max-threads should be set to "5". Creating lots of threads can hurt
throughput because the JVM spends more time managing them. I would
be
interested in hearing what others think.
Thinking about this more, there are some other things that need
to
be
fixed:
1. The JobPoller uses an unbounded queue. In a busy server, there is
the
potential the queue will grow in size until it causes an
out-of-memory
condition.
2. There is no accommodation for when a job cannot be added to the
queue
-
it is just lost. We could add a dequeue method to the Job interface
that
will allow implementations to recover or reschedule the job when it
can't
be added to the queue.
3. There is a JobPoller instance per delegator, and each instance
contains
the number of threads configured in serviceengine.xml. With the
current
max-threads setting of 15, a multi-tenant installation with 100
tenants
will create up to 1500 threads. (!!!) A smarter strategy might be to
have a
single JobPoller instance that services multiple JobManagers.
-Adrian
